Automated data center maintenance

ABSTRACT

Techniques for automated data center maintenance are described. In an example embodiment, an automated maintenance device may comprise processing circuitry and non-transitory computer-readable storage media comprising instructions for execution by the processing circuitry to cause the automated maintenance device to receive an automation command from an automation coordinator for a data center, identify an automated maintenance procedure based on the received automation command, and perform the identified automated maintenance procedure. Other embodiments are described and claimed.

RELATED CASE

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/365,969, filed Jul. 22, 2016, U.S. Provisional Patent ApplicationNo. 62/376,859, filed Aug. 18, 2016, and U.S. Provisional PatentApplication No. 62/427,268, filed Nov. 29, 2016, each of which is herebyincorporated by reference in its entirety.

BACKGROUND

In the course of ordinary operation of a data center, various types ofmaintenance are typically necessary in order to maintain desired levelsof performance, stability, and reliability. Examples of such maintenanceinclude testing, repair, replacement, and/or reconfiguration ofcomponents, installing new components, upgrading existing components,repositioning components and equipment, and other tasks of such anature. A large modern data center may contain great numbers ofcomponents and equipment of various types, and as a result, may have thepotential to impose a fairly substantial maintenance burden.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an embodiment of a first data center.

FIG. 2 illustrates an embodiment of a logical configuration of a rack.

FIG. 3 illustrates an embodiment of a second data center.

FIG. 4 illustrates an embodiment of a third data center.

FIG. 5 illustrates an embodiment of a connectivity scheme.

FIG. 6 illustrates an embodiment of first rack architecture.

FIG. 7 illustrates an embodiment of a first sled.

FIG. 8 illustrates an embodiment of a second rack architecture.

FIG. 9 illustrates an embodiment of a rack.

FIG. 10 illustrates an embodiment of a second sled.

FIG. 11 illustrates an embodiment of a fourth data center.

FIG. 12 illustrates an embodiment of a first logic flow.

FIG. 13 illustrates an embodiment of a fifth data center.

FIG. 14 illustrates an embodiment of an automated maintenance device.

FIG. 15 illustrates an embodiment of a first operating environment.

FIG. 16 illustrates an embodiment of a second operating environment.

FIG. 17 illustrates an embodiment of a third operating environment.

FIG. 18 illustrates an embodiment of a fourth operating environment.

FIG. 19 illustrates an embodiment of a fifth operating environment.

FIG. 20 illustrates an embodiment of a sixth operating environment.

FIG. 21 illustrates an embodiment of a first logic flow.

FIG. 22 illustrates an embodiment of a second logic flow.

FIG. 23 illustrates an embodiment of a third logic flow.

FIG. 24A illustrates an embodiment of a first storage medium.

FIG. 24B illustrates an embodiment of a second storage medium.

FIG. 25 illustrates an embodiment of a computing architecture.

FIG. 26 illustrates an embodiment of a communications architecture.

FIG. 27 illustrates an embodiment of a communication device.

FIG. 28 illustrates an embodiment of a first wireless network.

FIG. 29 illustrates an embodiment of a second wireless network.

DETAILED DESCRIPTION

Various embodiments may be generally directed to techniques forautomated data center maintenance. In one embodiment, for example, anautomated maintenance device may comprise processing circuitry andnon-transitory computer-readable storage media comprising instructionsfor execution by the processing circuitry to cause the automatedmaintenance device to receive an automation command from an automationcoordinator for a data center, identify an automated maintenanceprocedure based on the received automation command, and perform theidentified automated maintenance procedure. Other embodiments aredescribed and claimed.

Various embodiments may comprise one or more elements. An element maycomprise any structure arranged to perform certain operations. Eachelement may be implemented as hardware, software, or any combinationthereof, as desired for a given set of design parameters or performanceconstraints. Although an embodiment may be described with a limitednumber of elements in a certain topology by way of example, theembodiment may include more or less elements in alternate topologies asdesired for a given implementation. It is worthy to note that anyreference to “one embodiment” or “an embodiment” means that a particularfeature, structure, or characteristic described in connection with theembodiment is included in at least one embodiment. The appearances ofthe phrases “in one embodiment,” “in some embodiments,” and “in variousembodiments” in various places in the specification are not necessarilyall referring to the same embodiment.

FIG. 1 illustrates a conceptual overview of a data center 100 that maygenerally be representative of a data center or other type of computingnetwork in/for which one or more techniques described herein may beimplemented according to various embodiments. As shown in FIG. 1, datacenter 100 may generally contain a plurality of racks, each of which mayhouse computing equipment comprising a respective set of physicalresources. In the particular non-limiting example depicted in FIG. 1,data center 100 contains four racks 102A to 102D, which house computingequipment comprising respective sets of physical resources (PCRs) 105Ato 105D. According to this example, a collective set of physicalresources 106 of data center 100 includes the various sets of physicalresources 105A to 105D that are distributed among racks 102A to 102D.Physical resources 106 may include resources of multiple types, suchas—for example—processors, co-processors, accelerators,field-programmable gate arrays (FPGAs), memory, and storage. Theembodiments are not limited to these examples.

The illustrative data center 100 differs from typical data centers inmany ways. For example, in the illustrative embodiment, the circuitboards (“sleds”) on which components such as CPUs, memory, and othercomponents are placed are designed for increased thermal performance. Inparticular, in the illustrative embodiment, the sleds are shallower thantypical boards. In other words, the sleds are shorter from the front tothe back, where cooling fans are located. This decreases the length ofthe path that air must to travel across the components on the board.Further, the components on the sled are spaced further apart than intypical circuit boards, and the components are arranged to reduce oreliminate shadowing (i.e., one component in the air flow path of anothercomponent). In the illustrative embodiment, processing components suchas the processors are located on a top side of a sled while near memory,such as DIMMs, are located on a bottom side of the sled. As a result ofthe enhanced airflow provided by this design, the components may operateat higher frequencies and power levels than in typical systems, therebyincreasing performance. Furthermore, the sleds are configured to blindlymate with power and data communication cables in each rack 102A, 102B,102C, 102D, enhancing their ability to be quickly removed, upgraded,reinstalled, and/or replaced. Similarly, individual components locatedon the sleds, such as processors, accelerators, memory, and data storagedrives, are configured to be easily upgraded due to their increasedspacing from each other. In the illustrative embodiment, the componentsadditionally include hardware attestation features to prove theirauthenticity.

Furthermore, in the illustrative embodiment, the data center 100utilizes a single network architecture (“fabric”) that supports multipleother network architectures including Ethernet and Omni-Path. The sleds,in the illustrative embodiment, are coupled to switches via opticalfibers, which provide higher bandwidth and lower latency than typicaltwister pair cabling (e.g., Category 5, Category 5e, Category 6, etc.).Due to the high bandwidth, low latency interconnections and networkarchitecture, the data center 100 may, in use, pool resources, such asmemory, accelerators (e.g., graphics accelerators, FPGAs, ASICs, etc.),and data storage drives that are physically disaggregated, and providethem to compute resources (e.g., processors) on an as needed basis,enabling the compute resources to access the pooled resources as if theywere local. The illustrative data center 100 additionally receives usageinformation for the various resources, predicts resource usage fordifferent types of workloads based on past resource usage, anddynamically reallocates the resources based on this information.

The racks 102A, 102B, 102C, 102D of the data center 100 may includephysical design features that facilitate the automation of a variety oftypes of maintenance tasks. For example, data center 100 may beimplemented using racks that are designed to be robotically-accessed,and to accept and house robotically-manipulable resource sleds.Furthermore, in the illustrative embodiment, the racks 102A, 102B, 102C,102D include integrated power sources that receive a greater voltagethan is typical for power sources. The increased voltage enables thepower sources to provide additional power to the components on eachsled, enabling the components to operate at higher than typicalfrequencies. FIG. 2 illustrates an exemplary logical configuration of arack 202 of the data center 100. As shown in FIG. 2, rack 202 maygenerally house a plurality of sleds, each of which may comprise arespective set of physical resources. In the particular non-limitingexample depicted in FIG. 2, rack 202 houses sleds 204-1 to 204-4comprising respective sets of physical resources 205-1 to 205-4, each ofwhich constitutes a portion of the collective set of physical resources206 comprised in rack 202. With respect to FIG. 1, if rack 202 isrepresentative of—for example—rack 102A, then physical resources 206 maycorrespond to the physical resources 105A comprised in rack 102A. In thecontext of this example, physical resources 105A may thus be made up ofthe respective sets of physical resources, including physical storageresources 205-1, physical accelerator resources 205-2, physical memoryresources 205-3, and physical compute resources 205-5 comprised in thesleds 204-1 to 204-4 of rack 202. The embodiments are not limited tothis example. Each sled may contain a pool of each of the various typesof physical resources (e.g., compute, memory, accelerator, storage). Byhaving robotically accessible and robotically manipulable sledscomprising disaggregated resources, each type of resource can beupgraded independently of each other and at their own optimized refreshrate.

FIG. 3 illustrates an example of a data center 300 that may generally berepresentative of one in/for which one or more techniques describedherein may be implemented according to various embodiments. In theparticular non-limiting example depicted in FIG. 3, data center 300comprises racks 302-1 to 302-32. In various embodiments, the racks ofdata center 300 may be arranged in such fashion as to define and/oraccommodate various access pathways. For example, as shown in FIG. 3,the racks of data center 300 may be arranged in such fashion as todefine and/or accommodate access pathways 311A, 311B, 311C, and 311D. Insome embodiments, the presence of such access pathways may generallyenable automated maintenance equipment, such as robotic maintenanceequipment, to physically access the computing equipment housed in thevarious racks of data center 300 and perform automated maintenance tasks(e.g., replace a failed sled, upgrade a sled). In various embodiments,the dimensions of access pathways 311A, 311B, 311C, and 311D, thedimensions of racks 302-1 to 302-32, and/or one or more other aspects ofthe physical layout of data center 300 may be selected to facilitatesuch automated operations. The embodiments are not limited in thiscontext.

FIG. 4 illustrates an example of a data center 400 that may generally berepresentative of one in/for which one or more techniques describedherein may be implemented according to various embodiments. As shown inFIG. 4, data center 400 may feature an optical fabric 412. Opticalfabric 412 may generally comprise a combination of optical signalingmedia (such as optical cabling) and optical switching infrastructure viawhich any particular sled in data center 400 can send signals to (andreceive signals from) each of the other sleds in data center 400. Thesignaling connectivity that optical fabric 412 provides to any givensled may include connectivity both to other sleds in a same rack andsleds in other racks. In the particular non-limiting example depicted inFIG. 4, data center 400 includes four racks 402A to 402D. Racks 402A to402D house respective pairs of sleds 404A-1 and 404A-2, 404B-1 and404B-2, 404C-1 and 404C-2, and 404D-1 and 404D-2. Thus, in this example,data center 400 comprises a total of eight sleds. Via optical fabric412, each such sled may possess signaling connectivity with each of theseven other sleds in data center 400. For example, via optical fabric412, sled 404A-1 in rack 402A may possess signaling connectivity withsled 404A-2 in rack 402A, as well as the six other sleds 404B-1, 404B-2,404C-1, 404C-2, 404D-1, and 404D-2 that are distributed among the otherracks 402B, 402C, and 402D of data center 400. The embodiments are notlimited to this example.

FIG. 5 illustrates an overview of a connectivity scheme 500 that maygenerally be representative of link-layer connectivity that may beestablished in some embodiments among the various sleds of a datacenter, such as any of example data centers 100, 300, and 400 of FIGS.1, 3, and 4. Connectivity scheme 500 may be implemented using an opticalfabric that features a dual-mode optical switching infrastructure 514.Dual-mode optical switching infrastructure 514 may generally comprise aswitching infrastructure that is capable of receiving communicationsaccording to multiple link-layer protocols via a same unified set ofoptical signaling media, and properly switching such communications. Invarious embodiments, dual-mode optical switching infrastructure 514 maybe implemented using one or more dual-mode optical switches 515. Invarious embodiments, dual-mode optical switches 515 may generallycomprise high-radix switches. In some embodiments, dual-mode opticalswitches 515 may comprise multi-ply switches, such as four-ply switches.In various embodiments, dual-mode optical switches 515 may featureintegrated silicon photonics that enable them to switch communicationswith significantly reduced latency in comparison to conventionalswitching devices. In some embodiments, dual-mode optical switches 515may constitute leaf switches 530 in a leaf-spine architectureadditionally including one or more dual-mode optical spine switches 520.

In various embodiments, dual-mode optical switches may be capable ofreceiving both Ethernet protocol communications carrying InternetProtocol (IP packets) and communications according to a second,high-performance computing (HPC) link-layer protocol (e.g., Intel'sOmni-Path Architecture's, Infiniband) via optical signaling media of anoptical fabric. As reflected in FIG. 5, with respect to any particularpair of sleds 504A and 504B possessing optical signaling connectivity tothe optical fabric, connectivity scheme 500 may thus provide support forlink-layer connectivity via both Ethernet links and HPC links. Thus,both Ethernet and HPC communications can be supported by a singlehigh-bandwidth, low-latency switch fabric. The embodiments are notlimited to this example.

FIG. 6 illustrates a general overview of a rack architecture 600 thatmay be representative of an architecture of any particular one of theracks depicted in FIGS. 1 to 4 according to some embodiments. Asreflected in FIG. 6, rack architecture 600 may generally feature aplurality of sled spaces into which sleds may be inserted, each of whichmay be robotically-accessible via a rack access region 601. In theparticular non-limiting example depicted in FIG. 6, rack architecture600 features five sled spaces 603-1 to 603-5. Sled spaces 603-1 to 603-5feature respective multi-purpose connector modules (MPCMs) 616-1 to616-5.

Included among the types of sleds to be accommodated by rackarchitecture 600 may be one or more types of sleds that featureexpansion capabilities. FIG. 7 illustrates an example of a sled 704 thatmay be representative of a sled of such a type. As shown in FIG. 7, sled704 may comprise a set of physical resources 705, as well as an MPCM 716designed to couple with a counterpart MPCM when sled 704 is insertedinto a sled space such as any of sled spaces 603-1 to 603-5 of FIG. 6.Sled 704 may also feature an expansion connector 717. Expansionconnector 717 may generally comprise a socket, slot, or other type ofconnection element that is capable of accepting one or more types ofexpansion modules, such as an expansion sled 718. By coupling with acounterpart connector on expansion sled 718, expansion connector 717 mayprovide physical resources 705 with access to supplemental computingresources 705B residing on expansion sled 718. The embodiments are notlimited in this context.

FIG. 8 illustrates an example of a rack architecture 800 that may berepresentative of a rack architecture that may be implemented in orderto provide support for sleds featuring expansion capabilities, such assled 704 of FIG. 7. In the particular non-limiting example depicted inFIG. 8, rack architecture 800 includes seven sled spaces 803-1 to 803-7,which feature respective MPCMs 816-1 to 816-7. Sled spaces 803-1 to803-7 include respective primary regions 803-1A to 803-7A and respectiveexpansion regions 803-1B to 803-7B. With respect to each such sledspace, when the corresponding MPCM is coupled with a counterpart MPCM ofan inserted sled, the primary region may generally constitute a regionof the sled space that physically accommodates the inserted sled. Theexpansion region may generally constitute a region of the sled spacethat can physically accommodate an expansion module, such as expansionsled 718 of FIG. 7, in the event that the inserted sled is configuredwith such a module.

FIG. 9 illustrates an example of a rack 902 that may be representativeof a rack implemented according to rack architecture 800 of FIG. 8according to some embodiments. In the particular non-limiting exampledepicted in FIG. 9, rack 902 features seven sled spaces 903-1 to 903-7,which include respective primary regions 903-1A to 903-7A and respectiveexpansion regions 903-1B to 903-7B. In various embodiments, temperaturecontrol in rack 902 may be implemented using an air cooling system. Forexample, as reflected in FIG. 9, rack 902 may feature a plurality offans 919 that are generally arranged to provide air cooling within thevarious sled spaces 903-1 to 903-7. In some embodiments, the height ofthe sled space is greater than the conventional “1U” server height. Insuch embodiments, fans 919 may generally comprise relatively slow, largediameter cooling fans as compared to fans used in conventional rackconfigurations. Running larger diameter cooling fans at lower speeds mayincrease fan lifetime relative to smaller diameter cooling fans runningat higher speeds while still providing the same amount of cooling. Thesleds are physically shallower than conventional rack dimensions.Further, components are arranged on each sled to reduce thermalshadowing (i.e., not arranged serially in the direction of air flow). Asa result, the wider, shallower sleds allow for an increase in deviceperformance because the devices can be operated at a higher thermalenvelope (e.g., 250 W) due to improved cooling (i.e., no thermalshadowing, more space between devices, more room for larger heat sinks,etc.).

MPCMs 916-1 to 916-7 may be configured to provide inserted sleds withaccess to power sourced by respective power modules 920-1 to 920-7, eachof which may draw power from an external power source 921. In variousembodiments, external power source 921 may deliver alternating current(AC) power to rack 902, and power modules 920-1 to 920-7 may beconfigured to convert such AC power to direct current (DC) power to besourced to inserted sleds. In some embodiments, for example, powermodules 920-1 to 920-7 may be configured to convert 277-volt AC powerinto 12-volt DC power for provision to inserted sleds via respectiveMPCMs 916-1 to 916-7. The embodiments are not limited to this example.

MPCMs 916-1 to 916-7 may also be arranged to provide inserted sleds withoptical signaling connectivity to a dual-mode optical switchinginfrastructure 914, which may be the same as—or similar to—dual-modeoptical switching infrastructure 514 of FIG. 5. In various embodiments,optical connectors contained in MPCMs 916-1 to 916-7 may be designed tocouple with counterpart optical connectors contained in MPCMs ofinserted sleds to provide such sleds with optical signaling connectivityto dual-mode optical switching infrastructure 914 via respective lengthsof optical cabling 922-1 to 922-7. In some embodiments, each such lengthof optical cabling may extend from its corresponding MPCM to an opticalinterconnect loom 923 that is external to the sled spaces of rack 902.In various embodiments, optical interconnect loom 923 may be arranged topass through a support post or other type of load-bearing element ofrack 902. The embodiments are not limited in this context. Becauseinserted sleds connect to an optical switching infrastructure via MPCMs,the resources typically spent in manually configuring the rack cablingto accommodate a newly inserted sled can be saved.

FIG. 10 illustrates an example of a sled 1004 that may be representativeof a sled designed for use in conjunction with rack 902 of FIG. 9according to some embodiments. Sled 1004 may feature an MPCM 1016 thatcomprises an optical connector 1016A and a power connector 1016B, andthat is designed to couple with a counterpart MPCM of a sled space inconjunction with insertion of MPCM 1016 into that sled space. CouplingMPCM 1016 with such a counterpart MPCM may cause power connector 1016 tocouple with a power connector comprised in the counterpart MPCM. Thismay generally enable physical resources 1005 of sled 1004 to sourcepower from an external source, via power connector 1016 and powertransmission media 1024 that conductively couples power connector 1016to physical resources 1005.

Sled 1004 may also include dual-mode optical network interface circuitry1026. Dual-mode optical network interface circuitry 1026 may generallycomprise circuitry that is capable of communicating over opticalsignaling media according to each of multiple link-layer protocolssupported by dual-mode optical switching infrastructure 914 of FIG. 9.In some embodiments, dual-mode optical network interface circuitry 1026may be capable both of Ethernet protocol communications and ofcommunications according to a second, high-performance protocol. Invarious embodiments, dual-mode optical network interface circuitry 1026may include one or more optical transceiver modules 1027, each of whichmay be capable of transmitting and receiving optical signals over eachof one or more optical channels. The embodiments are not limited in thiscontext.

Coupling MPCM 1016 with a counterpart MPCM of a sled space in a givenrack may cause optical connector 1016A to couple with an opticalconnector comprised in the counterpart MPCM. This may generallyestablish optical connectivity between optical cabling of the sled anddual-mode optical network interface circuitry 1026, via each of a set ofoptical channels 1025. Dual-mode optical network interface circuitry1026 may communicate with the physical resources 1005 of sled 1004 viaelectrical signaling media 1028. In addition to the dimensions of thesleds and arrangement of components on the sleds to provide improvedcooling and enable operation at a relatively higher thermal envelope(e.g., 250 W), as described above with reference to FIG. 9, in someembodiments, a sled may include one or more additional features tofacilitate air cooling, such as a heatpipe and/or heat sinks arranged todissipate heat generated by physical resources 1005. It is worthy ofnote that although the example sled 1004 depicted in FIG. 10 does notfeature an expansion connector, any given sled that features the designelements of sled 1004 may also feature an expansion connector accordingto some embodiments. The embodiments are not limited in this context.

FIG. 11 illustrates an example of a data center 1100 that may generallybe representative of one in/for which one or more techniques describedherein may be implemented according to various embodiments. As reflectedin FIG. 11, a physical infrastructure management framework 1150A may beimplemented to facilitate management of a physical infrastructure 1100Aof data center 1100. In various embodiments, one function of physicalinfrastructure management framework 1150A may be to manage automatedmaintenance functions within data center 1100, such as the use ofrobotic maintenance equipment to service computing equipment withinphysical infrastructure 1100A. In some embodiments, physicalinfrastructure 1100A may feature an advanced telemetry system thatperforms telemetry reporting that is sufficiently robust to supportremote automated management of physical infrastructure 1100A. In variousembodiments, telemetry information provided by such an advancedtelemetry system may support features such as failureprediction/prevention capabilities and capacity planning capabilities.In some embodiments, physical infrastructure management framework 1150Amay also be configured to manage authentication of physicalinfrastructure components using hardware attestation techniques. Forexample, robots may verify the authenticity of components beforeinstallation by analyzing information collected from a radio frequencyidentification (RFID) tag associated with each component to beinstalled. The embodiments are not limited in this context.

As shown in FIG. 11, the physical infrastructure 1100A of data center1100 may comprise an optical fabric 1112, which may include a dual-modeoptical switching infrastructure 1114. Optical fabric 1112 and dual-modeoptical switching infrastructure 1114 may be the same as—or similarto—optical fabric 412 of FIG. 4 and dual-mode optical switchinginfrastructure 514 of FIG. 5, respectively, and may providehigh-bandwidth, low-latency, multi-protocol connectivity among sleds ofdata center 1100. As discussed above, with reference to FIG. 1, invarious embodiments, the availability of such connectivity may make itfeasible to disaggregate and dynamically pool resources such asaccelerators, memory, and storage. In some embodiments, for example, oneor more pooled accelerator sleds 1130 may be included among the physicalinfrastructure 1100A of data center 1100, each of which may comprise apool of accelerator resources—such as co-processors and/or FPGAs, forexample—that is available globally accessible to other sleds via opticalfabric 1112 and dual-mode optical switching infrastructure 1114.

In another example, in various embodiments, one or more pooled storagesleds 1132 may be included among the physical infrastructure 1100A ofdata center 1100, each of which may comprise a pool of storage resourcesthat is available globally accessible to other sleds via optical fabric1112 and dual-mode optical switching infrastructure 1114. In someembodiments, such pooled storage sleds 1132 may comprise pools ofsolid-state storage devices such as solid-state drives (SSDs). Invarious embodiments, one or more high-performance processing sleds 1134may be included among the physical infrastructure 1100A of data center1100. In some embodiments, high-performance processing sleds 1134 maycomprise pools of high-performance processors, as well as coolingfeatures that enhance air cooling to yield a higher thermal envelope ofup to 250 W or more. In various embodiments, any given high-performanceprocessing sled 1134 may feature an expansion connector 1117 that canaccept a far memory expansion sled, such that the far memory that islocally available to that high-performance processing sled 1134 isdisaggregated from the processors and near memory comprised on thatsled. In some embodiments, such a high-performance processing sled 1134may be configured with far memory using an expansion sled that compriseslow-latency SSD storage. The optical infrastructure allows for computeresources on one sled to utilize remote accelerator/FPGA, memory, and/orSSD resources that are disaggregated on a sled located on the same rackor any other rack in the data center. The remote resources can belocated one switch jump away or two-switch jumps away in the spine-leafnetwork architecture described above with reference to FIG. 5. Theembodiments are not limited in this context.

In various embodiments, one or more layers of abstraction may be appliedto the physical resources of physical infrastructure 1100A in order todefine a virtual infrastructure, such as a software-definedinfrastructure 1100B. In some embodiments, virtual computing resources1136 of software-defined infrastructure 1100B may be allocated tosupport the provision of cloud services 1140. In various embodiments,particular sets of virtual computing resources 1136 may be grouped forprovision to cloud services 1140 in the form of SDI services 1138.Examples of cloud services 1140 may include—without limitation—softwareas a service (SaaS) services 1142, platform as a service (PaaS) services1144, and infrastructure as a service (IaaS) services 1146.

In some embodiments, management of software-defined infrastructure 1100Bmay be conducted using a virtual infrastructure management framework1150B. In various embodiments, virtual infrastructure managementframework 1150B may be designed to implement workload fingerprintingtechniques and/or machine-learning techniques in conjunction withmanaging allocation of virtual computing resources 1136 and/or SDIservices 1138 to cloud services 1140. In some embodiments, virtualinfrastructure management framework 1150B may use/consult telemetry datain conjunction with performing such resource allocation. In variousembodiments, an application/service management framework 1150C may beimplemented in order to provide QoS management capabilities for cloudservices 1140. The embodiments are not limited in this context.

FIG. 12 illustrates an example of a logic flow 1200 that may berepresentative of a maintenance algorithm for a data center, such as oneor more of data center 100 of FIG. 1, data center 300 of FIG. 3, datacenter 400 of FIG. 4, and data center 1100 of FIG. 11. As shown in FIG.12, data center operation information may be collected at 1202. Invarious embodiments, the collected data center operation information mayinclude information describing various characteristics of ongoingoperation of the data center, such as resource utilization levels,workload sizes, throughput rates, temperature measurements, and soforth. In some embodiments, the collected data center operationinformation may additionally or alternatively include informationdescribing other characteristics of the data center, such as the typesof resources comprised in the data center, the locations/distributionsof such resources within the data center, the capabilities and/orfeatures of those resources, and so forth. The embodiments are notlimited to these examples.

Based on data center operation information such as may be collected at1202, a maintenance task to be completed may be identified at 1204. Inone example, based on data center operation information indicating thatprocessing resources on a given sled are non-responsive tocommunications from resources on other sleds, it may be determined at1204 that the sled is to be pulled for testing. In another example,based on data center operation information indicating that a particularDIMM has reached the end of its estimated service life, it may bedetermined that the DIMM is to be replaced. At 1206, a set of physicalactions associated with the maintenance task may be determined, andthose physical actions may be performed at 1208 in order to complete themaintenance task. For instance, in the aforementioned example in whichit is determined at 1204 that a DIMM is to be replaced, the physicalactions identified at 1206 and performed at 1208 may include travelingto a particular rack in order to access a sled comprising the DIMM,removing the DIMM from a socket on the sled, and inserting a replacementDIMM into the socket. The embodiments are not limited to this example.

FIG. 13 illustrates an overhead view of an example data center 1300.According to various embodiments, data center 1300 may be representativeof a data center in which various operations associated with data centermaintenance—such as operations associated with one or more of blocks1202, 1204, 1206, and 1208 in logic flow 1200 of FIG. 12—are automatedusing the capabilities of robotic maintenance equipment. According tosome embodiments, data center 1300 may be representative of one or moreof data center 100 of FIG. 1, data center 300 of FIG. 3, data center 400of FIG. 4, and data center 1100 of FIG. 11. The embodiments are notlimited in this context.

In various embodiments, according to an automated maintenance schemeimplemented in data center 1300, robots 1360 may be used to service,repair, replace, clean, test, configure, upgrade, move, position, and/orotherwise manipulate equipment housed in racks 1302. Racks 1302 may bearranged in such fashion as to define and/or accommodate access pathwaysvia which robots 1360 can physically access such equipment. Robots 1360may traverse such access pathways in conjunction with moving around indata center 1300 to perform various tasks. Physical features ofequipment housed in racks 1302 may be designed to facilitate roboticmanipulation/handling. It is to be appreciated that in variousembodiments, the equipment housed in racks 1302 may include someequipment that is not robotically accessible/serviceable. Further, insome embodiments, there may be some equipment within data center 1300that is robotically accessible/serviceable but is not housed in racks1302. The embodiments are not limited in this context.

FIG. 14 illustrates a block diagram of an automated maintenance device1400 that may be representative of any given robot 1360 in data center1300 of FIG. 13 according to various embodiments. As shown in FIG. 14,automated maintenance device 1400 may comprise a variety of elements. Inthe non-limiting example depicted in FIG. 14, automated maintenancedevice 1400 comprises locomotion elements 1462, manipulation elements1463, sensory elements 1464, communication elements 1465, interfaces1466, memory/storage elements 1467, and operations management andcontrol (OMC) elements 1468.

Locomotion elements 1462 may generally comprise physical elementsenabling automated maintenance device 1400 to move around within a datacenter. In various embodiments, locomotion elements 1462 may comprisewheels. In some embodiments, locomotion elements 1462 may comprisecaterpillar tracks. In various embodiments, automated maintenance device1400 may provide the motive power/force required for motion. Forexample, in some embodiments, automated maintenance device 1400 mayfeature a battery that provides power to drive wheels or tracks used byautomated maintenance device 1400 for moving around in a data center. Invarious other embodiments, the motive power/force may be provided by anexternal source. The embodiments are not limited in this context.

Manipulation elements 1463 may generally comprise physical elements thatare usable to manipulate various types of equipment in a data center. Insome embodiments, manipulation elements 1463 may include one or morerobotic arms. In various embodiments, manipulation elements 1463 mayinclude one or more multi-link manipulators. In some embodiments,manipulation elements 1463 may include one or more end effectors usablefor gripping various types of equipment, components, and/or otherobjects within the data center. In various embodiments, manipulationelements 1463 may include one or more end effectors comprising impactivegrippers, such as jaw or claw grippers. In some embodiments,manipulation elements 1463 may include one or more end effectorscomprising ingressive grippers, which may feature pins, needles,hackles, or other elements that are to physically penetrate the surfaceof an object being gripped. In various embodiments, manipulationelements 1463 may include one or more end effectors comprisingastrictive grippers, which may grip objects using air suction, magneticadhesion, or electroadhesion. The embodiments are not limited to theseexamples.

Sensory elements 1464 may generally comprise physical elements that areusable to sense various aspects of ambient conditions within a datacenter. Examples of sensory elements 1464 may include cameras, alignmentguides/sensors, distance sensors, proximity sensors, barcode readers,RFID/NFC readers, temperature sensors, airflow sensors, air qualitysensors, humidity sensors, and pressure sensors. The embodiments are notlimited to these examples.

Communication elements 1465 may generally comprise a set of electroniccomponents and/or circuitry operable to perform functions associatedwith communications between automated maintenance device 1400 and one ormore external devices. In a given embodiment, such communications mayinclude wireless communications, wired communications, or both. Invarious embodiments, communication elements 1465 may include elementsoperative to generate/construct packets, frames, messages, and/or otherinformation to be wirelessly communicated to external device(s), and/orto process/deconstruct packets, frames, messages, and/or otherinformation wirelessly received from external device(s). In variousembodiments, for example, communication elements 1465 may includebaseband circuitry supporting wireless communications according to oneor more wireless communication protocols/standards. In some embodiments,communication elements 1465 may include elements operative to generate,process, construct, and/or deconstruct packets, frames, messages, and/orother information communicated over wired media. In various embodiments,for example, communication elements 1465 may include network interfacecircuitry supporting wired communications according to one or more wiredcommunication protocols/standards. The embodiments are not limited inthis context.

In various embodiments, interfaces 1466 may include one or morecommunication interfaces 1466A. As reflected in FIG. 14, examples ofinterfaces 1466 that automated maintenance device 1400 may feature invarious embodiments may include—without limitation—communicationinterfaces 1466A, testing interfaces 1466B, power interfaces 1466C, anduser interfaces 1466D.

Communication interfaces 1466A may generally comprise interfaces usableto transmit and/or receive signals via one or more communication media,which may include wired media, wireless media, or both. In variousembodiments, communication interfaces 1466A may include one or morewireless communication interfaces, such as radio frequency (RF)interfaces and/or optical wireless communication (OWC) interfaces. Insome embodiments, communication interfaces may additionally oralternatively include one or more wired communication interfaces, suchas interface(s) for communicating over media such as coaxial cable,twisted pair, and optical fiber. The embodiments are not limited tothese examples.

In various embodiments, interfaces 1466 may include one or more testinginterfaces 1466B. Testing interfaces 1466B may generally compriseinterfaces via which automated maintenance device 1400 is able to testphysical components/resources of one or more types, which mayinclude—without limitation—one or more of physical storage resources205-1, physical accelerator resources 205-2, physical memory resources205-3, and physical compute resources 205-4 of FIG. 2. In an exampleembodiment, interfaces 1466 may include a testing interface 1466B thatenables automated maintenance device 1400 to test the functionality of aDIMM inserted into a testing slot. The embodiments are not limited tothese examples.

In various embodiments, interfaces 1466 may include one or more powerinterfaces 1466C. Power interfaces 1466C may generally compriseinterfaces via which automated maintenance device 1400 can draw and/orsource power. In various embodiments, power interfaces 1466C may includeone or more interfaces via which automated maintenance device 1400 candraw power from external source(s). In some embodiments, automatedmaintenance device 1400 may feature one or more power interfaces 1466Cconfigured to provide charge to one or more batteries (not shown), andautomated maintenance device may draw its operating power from those oneor more batteries. In various embodiments, automated maintenance device1400 may feature one or more power interfaces 1466C via which it candirectly draw operating power. In various embodiments, automatedmaintenance device 1400 may feature one or more power interfaces 1466Cvia which it can source power to external devices. For example, invarious embodiments, automated maintenance device 1400 may feature apower interface 1466C via which it can source power to charge a batteryof a second automated maintenance device. The embodiments are notlimited to this example.

In some embodiments, interfaces 1466 may include one or more userinterfaces. User interfaces 1466D may generally comprise interfaces viawhich information can be provided to human technicians and/or user inputcan be accepted from human technicians. Examples of user interfaces1466D may include displays, touchscreens, speakers, microphones,keypads, mice, trackballs, trackpads, joysticks, fingerprint readers,retinal scanners, buttons, switches, and the like. The embodiments arenot limited to these examples.

Memory/storage elements 1467 may generally comprise a set of electroniccomponents and/or circuitry capable of retaining data, such as any ofvarious types of data that may be generated, transmitted, received,and/or used by automated maintenance device 1400 during normaloperation. In some embodiments, memory/storage elements 1467 may includeone or both of volatile memory and non-volatile memory. For example, invarious embodiments, memory/storage elements 1467 may include one ormore of read-only memory (ROM), random-access memory (RAM), dynamic RAM(DRAM), Double-Data-Rate DRAM (DDRAM), synchronous DRAM (SDRAM), staticRAM (SRAM), programmable ROM (PROM), erasable programmable ROM (EPROM),electrically erasable programmable ROM (EEPROM), flash memory, polymermemory such as ferroelectric polymer memory, ovonic memory, phase changeor ferroelectric memory, silicon-oxide-nitride-oxide-silicon (SONOS)memory, magnetic or optical cards, hard disks, an array of devices suchas Redundant Array of Independent Disks (RAID) drives, solid statememory devices, solid state drives (SSDs), or any other type of mediasuitable for storing information. The embodiments are not limited tothese examples.

OMC elements 1468 may generally comprise a set of components and/orcircuitry capable of performing computing operations required toimplement logic for managing and controlling the operations of automatedmaintenance device 1400. In various embodiments, OMC elements 1468 mayinclude processing circuitry, such as one or more processors/processingunits. In some embodiments, an automation engine 1469 may execute onsuch processing circuitry. Automation engine 1469 may generally beoperative to conduct overall management, control, coordination, and/oroversight of the operations of automated maintenance device 1400. Invarious embodiments, this may include management, coordination, control,and/or oversight of the operations/usage of various other elementswithin automated maintenance device 1400, such as any or all oflocomotion elements 1462, manipulation elements 1463, sensory elements1464, communication elements 1465, interfaces 1466, and memory/storageelements 1467. The embodiments are not limited in this context.

FIG. 15 illustrates an example of an operating environment 1500 that maybe representative of the implementation of an automated maintenancescheme in data center 1300 according to various embodiments. Accordingto such an automated maintenance scheme, an automation coordinator 1555may centrally manage/coordinate various aspects of automated maintenanceoperations in data center 1300. In some embodiments, automationcoordinator 1555 may centrally manage/coordinate various aspects ofautomated maintenance operations in data center 1300 based in part ontelemetry data 1571 provided by a telemetry framework 1570. According tovarious embodiments, telemetry framework 1570 may be representative ofan advanced telemetry system that performs telemetry reporting forphysical infrastructure 1100A in data center 1100 of FIG. 11, andautomation coordinator 1555 may be representative of automatedmaintenance coordination functionality of physical infrastructuremanagement framework 1150A. The embodiments are not limited in thiscontext.

In some embodiments, management/coordination functionality of automationcoordinator 1555 may be provided by a coordination engine 1572. Invarious embodiments, coordination engine 1572 may execute on processingcircuitry of automation coordinator 1555. In various embodiments,coordination engine 1572 may generate automation commands 1573 fortransmission to robots 1360 in order to instruct robots 1360 to performautomated maintenance tasks and/or actions associated with such tasks.In some embodiments, robots 1360 may provide automation coordinator 1555with various types of feedback 1574 in order to—for example—acknowledgeautomation commands 1573, report the results of attempted maintenancetasks, provide information regarding the statuses of components,resources, and/or equipment, provide information regarding informationregarding the statuses of robots 1360 themselves, and/or reportmeasurements of one or more aspects of ambient conditions in the datacenter. The embodiments are not limited to these examples.

In some embodiments, coordination engine 1572 may consider various typesof information in conjunction with automated maintenancecoordination/management. As reflected in FIG. 15, examples of such typesof information may include physical infrastructure information 1575,data center operations information 1576, maintenance task information1577, and maintenance equipment information 1579.

Physical infrastructure information 1575 may generally compriseinformation identifying equipment, devices, components, interconnects,physical resources, and/or other infrastructure elements that compriseportions of the physical infrastructure of data center 1300, anddescribing characteristics of such elements. Data center operationsinformation 1576 may generally comprise information describing variousaspects of ongoing operations within data center 1300. In someembodiments, for example, data center operations information 1576 mayinclude information describing one or more workloads currently beingprocessed in data center 1300. In various embodiments, data centeroperations information 1576 may include metrics characterizing one ormore aspects of current operations in data center 1300. For example, insome embodiments, data center operations information 1576 may includeperformance metrics characterizing the relative level of performancecurrently being achieved in data center 1300, efficiency metricscharacterizing the relative level of efficiency with which the physicalresources of data center 1300 are being used to handle the currentworkloads, and utilization metrics generally indicative of current usagelevels of various types of resources in data center 1300. In variousembodiments, data center operations information 1576 may includetelemetry data 1571, such as automation coordinator 1555 may receive viatelemetry framework 1570 or from robots 1360. The embodiments are notlimited in this context.

Maintenance task information 1577 may generally comprise informationidentifying and describing ongoing and pending maintenance tasks of datacenter 1300. Maintenance task information 1577 may also includeinformation identifying and describing previously completed maintenancetasks. In various embodiments, maintenance task information 1577 mayinclude a pending task queue 1578. Pending task queue 1578 may generallycomprise information identifying a set of maintenance tasks that need tobe performed in data center 1300. Maintenance equipment information 1579may generally comprise identifying and describing automated maintenanceequipment—such as robots 1360—of data center 1300. In some embodiments,maintenance equipment information 1579 may include a candidate devicepool 1580. Candidate device pool 1580 may generally comprise informationidentifying a set of robots 1360 that are currently available for use indata center 1300. The embodiments are not limited in this context.

In various embodiments, based on telemetry data 1571, automationcoordinator 1555 may identify automated maintenance tasks to beperformed in data center 1300 by robots 1360. For example, based ontelemetry data 1571 indicating a high bit error rate at a DIMM,automation coordinator 1555 may determine that a robot 1360 should beassigned to replace that DIMM. In some embodiments, automationcoordinator 1555 may use telemetry data 1571 to prioritize amongautomated maintenance tasks, such as tasks comprised in pending taskqueue 1578. For example, automation coordinator 1555 may use telemetrydata 1571 to assess the respective expected performance impacts ofmultiple automated maintenance tasks in pending task queue 1578, and mayassign out an automated maintenance task with the highest expectedperformance impact first. In some embodiments, in identifying and/orprioritizing among automated maintenance tasks, automation coordinator1555 may consider any or all of physical infrastructure information1575, data center operations information 1576, maintenance taskinformation 1577, and maintenance equipment information 1579 in additionto—or in lieu of—telemetry data 1571.

In a first example, automation coordinator 1555 may assign a lowpriority to an automated maintenance task involving replacement of amalfunctioning compute sled based on physical infrastructure information1575 indicating that another sled in a different rack can be used as asubstitute without need for replacing the malfunctioning compute sled.In a second example, automation coordinator 1555 may assign a highpriority to an automated maintenance task involving replacing amalfunctioning memory sled based on data center operation information1576 indicating that a scarcity of memory constitutes a performancebottleneck with respect to workloads being processed in data center1300. In a third example, automation coordinator 1555 may determine notto add a new maintenance task to pending task queue 1578 based on adetermination that a maintenance task already present in pending taskqueue 1578 may render the new maintenance task unnecessary and/or moot.In a fourth example, in determining an extent to which to prioritize anautomated maintenance task that requires the use of particular robots1360 featuring specialized capabilities, automation coordinator 1555 mayconsider maintenance equipment information 1579 indicating whether anyrobots 1360 featuring such specialized capabilities are currentlyavailable. The embodiments are not limited to these examples.

In various embodiments, based on telemetry data 1571, automationcoordinator 1555 may control the positioning and/or movement of robots1360 within data center 1300. For example, having used telemetry data1571 to identify a region of data center 1300 within which a greaternumber of hardware failures have been and/or are expected to beobserved, automation coordinator 1555 may position robots 1360 moredensely within that identified region than within other regions of datacenter 1300. The embodiments are not limited in this context.

In some embodiments, in response to automated maintenance decisions—suchas may be reached based on any or all of telemetry data 1571, physicalinfrastructure information 1575, data center operations information1576, maintenance task information 1577, and maintenance equipmentinformation 1579—automation coordinator 1555 may send automationcommands 1573 to robots 1360 in order to instruct robots 1360 to performoperations associated with automated maintenance tasks. For example,upon determining that a particular compute sled should be replaced,automation coordinator 1555 may send an automation command 1573 in orderto instruct a robot 1360 to perform a sled replacement procedure toreplace the sled. In various embodiments, automation coordinator 1555may inform robots 1360 of various parameters characterizing assignedautomated maintenance tasks by including such parameters in automationcommands 1573. For instance, in the context of the preceding example,the automation command 1573 may contain fields specifying a sled IDuniquely identifying the sled to be replaced and a rack ID and/or sledspace ID identifying the location of that sled within the data center,as well as analogous parameters associated with the replacement sled.The embodiments are not limited to this example.

It is worthy of note that in various embodiments, with respect to someaspects of automated maintenance operations, decision-making may behandled in a distributed—rather than centralized—fashion. In suchembodiments, robots 1360 may make some automated maintenance decisionsautonomously. In some such embodiments, as illustrated in FIG. 15,robots 1360 may perform such autonomous decision-making based ontelemetry data 1571 received from telemetry framework 1570. In anexample embodiment, a robot 1360 may determine based on analysis oftelemetry data 1571 that a particular CPU is malfunctioning, andautonomously decide to replace that malfunctioning CPU. In variousembodiments, some or all of the robots 1360 in data center 1300 may haveaccess to any or all of physical infrastructure information 1575, datacenter operations information 1576, maintenance task information 1577,and maintenance equipment information 1579, and may consider suchinformation as well in conjunction with autonomous decision-making. Invarious embodiments, distributed coordination functions may beimplemented to enable some types of maintenance tasks to be completedvia collaborative maintenance procedures involving cooperation betweenmultiple robots. The embodiments are not limited in this context.

FIG. 16 illustrates an example of an operating environment 1600 that maybe representative of various embodiments. In operating environment 1600,in conjunction with automated maintenance operations in data center1300, robots 1360 may provide automation coordinator 1555 with feedback1574 that includes one or more of position data 1681, assistance data1682, and environmental data 1683. The embodiments are not limited tothese examples. It is worthy of note that in some embodiments, althoughnot depicted in FIG. 16, robots 1360 may gather various types oftelemetry data 1571 in conjunction with automated maintenance operationsand include such gathered telemetry data 1571 in the feedback 1574provided to automation coordinator 1555. The embodiments are not limitedin this context.

Position data 1681 may generally comprise data for use by automationcoordinator 1555 to determine/track the positions and/or movements ofrobots 1360 within data center 1300. In some embodiments, position data1681 may comprise data associated with an indoor positioning system. Insome such embodiments, the indoor positioning system may be aradio-based system, such as a Wi-Fi-based or Bluetooth-based indoorpositioning system. In some other embodiments, a non-radio basedpositioning system, such as a magnetic, optical, or inertial indoorpositioning system may be used. In various embodiments, the indoorpositioning system may be a hybrid system, such as one that combines twoor more of radio-based, magnetic, optical, and inertial indoorpositioning techniques. The embodiments are not limited in this context.

Assistance data 1682 may generally comprise data for use by automationcoordinator 1555 to provide human maintenance personnel with informationaiding them in the identification and/or performance of manualmaintenance tasks. In various embodiments, a given robot 1360 maygenerate assistance data 1682 in response to identifying a maintenanceissue that it cannot correct/resolve in an automated fashion. Forinstance, after identifying a component that needs to be replaced anddetermining that it cannot perform the replacement itself, a robot 1360take a picture of the component and provide assistance data 1682comprising that picture to automation coordinator 1555. Automationcoordinator 1555 may then cause the picture to be presented on a displayfor reference by human maintenance personnel in order to aid visualidentification of the component to be replaced. The embodiments are notlimited to this example.

In some embodiments, the performance and/or reliability of various typesof hardware in data center 1300 may potentially be affected by one ormore aspects of the ambient conditions within data center 1300, such asambient temperature, pressure, humidity, and air quality. For example, arate at which corrosion occurs on metallic contacts of components suchas DIMMs may depend on the ambient temperature and humidity. In variousembodiments, it may thus be desirable to monitor various types ofenvironmental parameters at various locations during ongoing operationsof data center 1300.

In some embodiments, robots 1360 may be configured to supportenvironmental condition monitoring by measuring one or more aspects ofambient conditions within the data center during ongoing operations andproviding those collected measurements to automation coordinator 1555 inthe form of environmental data 1683. In various embodiments, robots 1360may collect environmental data 1683 using sensors or sensor arrayscomprising sensory elements such as sensory elements 1464 of FIG. 14.Examples of conditions/parameters that robots 1360 may measure andreport to automation coordinator 1555 in the form of environmental data1683 may include—without limitation—temperature, pressure, humidity, andair quality. In some embodiments, in conjunction with providingenvironmental condition measurements in the form of environmental data1683, robots 1360 may also provide corresponding position data 1681 thatindicates the locations at which the associated measurements wereperformed. The embodiments are not limited in this context.

In various embodiments, access to dynamic, continuous, andlocation-specific measurements of such parameters may enable a datacenter operator to predict failures, dynamically configure systems forbest performance, and dynamically move resources for data centeroptimization. In some embodiments, based on environmental data 1683provided by robots 1360, a data center operator may be able to predictaccelerated failure of parts versus standard factory specification andreplace parts earlier (or move to lower priority tasks). In variousembodiments, environmental data 1683 provided by robots 1360 may enablea data center operator to initiate service tickets ahead of predictedfailure timelines. For example, a cleaning of DIMM contacts may beinitiated in order to avoid corrosion build-up to the level wherefailures start occurring. In some embodiments, environmental data 1683provided by robots 1360 may enable a data center operator tocontinuously and dynamically configure servers based on, for example,altitude, pressure and other parameters that may be important to suchthings as fan speeds and cooling configurations which in turn may affectperformance of a server in a given environment and temperature. Invarious embodiments, environmental data 1683 provided by robots 1360 mayenable a data center operator to detect and move data center resourcesautomatically from zones/locations of the data center that may beaffected by equipment failures or environment variations detected by therobot's sensors. For example, based on environmental data 1683indicating an excessive temperature or air quality deterioration in aparticular data center region, servers and/or other resources may berelocated from the affected region to a different region. Theembodiments are not limited to these examples.

FIG. 17 illustrates an example of an operating environment 1700 that maybe representative of the implementation of an automated data centermaintenance scheme according to some embodiments. In operatingenvironment 1700, a robot 1760 may perform one or more automatedmaintenance tasks at a rack 1702. According to some embodiments, robot1760 may be representative of a robot 1360 that performs operationsassociated with automated data center maintenance in data center 1300 ofFIGS. 13, 15, and 16. In various embodiments, robot 1760 may beimplemented using automated maintenance device 1400 of FIG. 14. Invarious embodiments, as reflected by the dashed line in FIG. 17, robot1760 may move to a location of rack 1702 from another location in orderto perform one or more automated maintenance tasks at rack 1702. In someembodiments, robot 1760 may perform one or more such tasks based onautomation commands 1773 received from automation coordinator 1555. Invarious embodiments, robot 1760 may additionally or alternativelyperform one or more such tasks autonomously, without intervention on thepart of automation coordinator 1555. The embodiments are not limited inthis context.

In some embodiments, robot 1760 may perform one or more automatedmaintenance tasks involving the installation and/or removal of sleds atracks of a data center such as data center 1300. In various embodiments,for example, robot 1760 may be operative to install a sled 1704 at rack1702. In some embodiments, robot 1760 may install sled 1704 by insertingit into an available sled space of rack 1702. In various embodiments, inconjunction with inserting sled 1704, robot 1760 may grip particularphysical elements designed to accommodate robotic manipulation/handling.In some embodiments, robot 1760 may use image recognition and/or otherlocation techniques to locate the elements to be gripped, and may insertsled 1704 while gripping those elements. In various embodiments, ratherthan installing sled 1704, robot 1760 may instead remove sled 1704 fromrack 1702 and install a replacement sled 1704B. In some embodiments,robot 1760 may install replacement sled 1704B in a same sled space aswas occupied by sled 1704, once it has removed sled 1704. In variousother embodiments, robot 1760 may install replacement sled 1704B in adifferent sled space, such that it does not need to remove sled 1704before installing replacement sled 1704B. The embodiments are notlimited in this context.

In some embodiments, robot 1760 may perform one or more automatedmaintenance tasks involving upkeep, repair, and/or replacement ofparticular components on sleds of a data center such as data center1300. In various embodiments, robot 1760 may be used to power up acomponent 1706 in accordance with a scheme for periodically powering upcomponents in the data center on a periodic basis in order to improvethe reliability of such components. In some embodiments, for example,storage and/or memory components may tend to malfunction when left idlefor excessive periods of time, and thus robots may be used to power upsuch components according to a defined cycle. In such an embodiment,robot 1760 may be operative to power up an appropriate component 1706 byplugging that component 1706 into a powered interface/slot. Theembodiments are not limited to this example.

In various embodiments, robot 1760 may be operative to manipulate agiven component 1706 in accordance with a scheme for automated upkeep ofpooled memory resources of a data center. According to such a scheme,robots may be used to assess/troubleshoot apparently malfunctioningmemory resources such as DIMMs. In some embodiments, according to such ascheme, robot 1760 may identify a component 1706 comprising a memoryresource such as a DIMM, remove that component 1706 from a slot on sled1704, and clean the component 1706. Robot 1760 may then test thecomponent 1706 to determine whether the issue has been resolved, and maydetermine to pull sled 1704 for “back-room” servicing if it finds thatthe problem persists. In various embodiments, robot 1760 may test thecomponent 1706 after reinserting it into its slot on sled 1704. In someother embodiments, robot 1760 may be configured with a testing slot intowhich it can insert the component 1706 for the purpose of testing. Theembodiments are not limited in this context.

FIG. 18 illustrates an example of an operating environment 1800 that maybe representative of the implementation of an automated data centermaintenance scheme according to some embodiments. In operatingenvironment 1800, a robot 1860 may perform automated CPU cache servicingfor a sled 1804 at a rack 1802. According to some embodiments, robot1860 may be representative of a robot 1360 that performs operationsassociated with automated data center maintenance in data center 1300 ofFIGS. 13, 15, and 16. In various embodiments, robot 1860 may beimplemented using automated maintenance device 1400 of FIG. 14. In someembodiments, as reflected by the dashed line in FIG. 18, robot 1860 maymove to a location of rack 1802 from another location in order toperform the automated CPU cache servicing for sled 1804. In variousembodiments, robot 1860 may perform such automated CPU cache servicingbased on automation commands 1873 received from automation coordinator1555. In some other embodiments, robot 1860 may perform the automatedCPU cache servicing autonomously, without intervention on the part ofautomation coordinator 1555. The embodiments are not limited in thiscontext.

As shown in FIG. 18, sled 1804 may comprise components 1806 that includea CPU 1806A, cache memory 1806B for the CPU 1806A, and a heatsink 1806Cfor the CPU 1806A. In various embodiments, cache memory 1806B mayunderlie CPU 1806A, and CPU 1806A may underlie heatsink 1806C. In someembodiments, cache memory 1806B may comprise one or more cache memorymodules. In various embodiments, the automated CPU cache servicing thatrobot 1860 performs in operating environment 1800 may involve replacingcache memory 1806B. For example, in some embodiments, cache memory 1806Bmay comprise one or more cache memory modules that robot 1860 removesfrom sled 1804 and replaces with one or more replacement cache modules.In various embodiments, the determination to perform automated CPU cacheservicing and thus replace cache memory 1806B may be based on adetermination that cache memory 1806B is not functioning properly or isoutdated. For example, in some embodiments, automation coordinator 1555may determine—based on telemetry data 1571 of FIG. 15—that cache memory1806B is not functioning, and may use robot 1860 to replace cache memory1806B in response to that determination. The embodiments are not limitedto this example.

In various embodiments, according to a procedure for automated CPU cacheservicing, robot 1860 may remove CPU 1806A and heat sink 1806C from sled1804 in order to gain physical access to cache memory 1806B. In someembodiments, robot 1860 may remove sled 1804 from rack 1802 prior toremoving CPU 1806A and heat sink 1806C from sled 1804. In various otherembodiments, robot 1860 may remove CPU 1806A and heat sink 1806C fromsled 1804 while sled 1804 remains seated within a sled space of rack1802. In some embodiments, robot 1860 may first remove heat sink 1806C,and then remove CPU 1806A. In various other embodiments, robot 1860 mayremove both heat sink 1806C and CPU 1806A simultaneously and/or as acollective unit (i.e., without removing heat sink 1806C from CPU 1806A).In some embodiments, after replacing cache memory 1806B, robot 1860 mayreinstall CPU 1806A and heat sink 1806C upon sled 1804, which it maythen reinsert into a sled space of rack 1802 in embodiments in which itwas previously removed. The embodiments are not limited in this context.

FIG. 19 illustrates an example of an operating environment 1900 that maybe representative of the implementation of an automated data centermaintenance scheme according to some embodiments. In operatingenvironment 1900, a robot 1960 may perform automated storage and/ortransfer of a compute state of a compute sled 1904 at a rack 1902.According to some embodiments, robot 1760 may be representative of arobot 1360 that performs operations associated with automated datacenter maintenance in data center 1300 of FIGS. 13, 15, and 16. Invarious embodiments, robot 1960 may be implemented using automatedmaintenance device 1400 of FIG. 14. In some embodiments, as reflected bythe dashed line in FIG. 19, robot 1960 may move to a location of rack1902 from another location in order to perform the automated storageand/or transfer of the compute state of compute sled 1904. In variousembodiments, robot 1960 may perform such automated compute state storageand/or transfer based on automation commands 1973 received fromautomation coordinator 1555. In some other embodiments, robot 1960 mayperform the automated compute state storage and/or transferautonomously, without intervention on the part of automation coordinator1555. The embodiments are not limited in this context.

As shown in FIG. 19, compute sled 1904 may comprise components 1906 thatinclude one or more CPUs 1906A and a connector 1906B. In variousembodiments, compute sled 1904 may comprise two CPUs 1906A. In someother embodiments, compute sled 1904 may comprise more than two CPUs1906A, or only a single CPU 1906A. Connector 1906B may generallycomprise a slot, socket, or other connective component designed toaccept a memory daughter card for use to store a compute state ofcompute sled 1904. In various embodiments, compute sled 1904 maycomprise two CPUs 1906A and connector 1906B may be located between thosetwo CPUs 1906A. The embodiments are not limited in this context.

In some embodiments, according to a procedure for automated computestate storage and/or transfer, robot 1960 may insert a memory card 1918into connector 1906B. In various embodiments, robot 1960 may removecompute sled 1904 from rack 1902 prior to inserting memory card 1918into connector 1906B. In some other embodiments, robot 1960 may insertmemory card 1918 into connector 1906B while compute sled 1904 remainsseated within a sled space of rack 1902. In still other embodiments,memory card 1918 may be present and coupled with connector 1906B priorto initiation of the automated compute state storage and/or transferprocedure. In various embodiments, memory card 1918 may comprise a setof physical memory resources 1906C. In some embodiments, once memorycard is inserted into/coupled with connector 1906B, a compute state 1984of compute sled 1904 may be stored on memory card 1918 using one or moreof the physical memory resources 1906C comprised thereon. In variousembodiments, compute state 1984 may include respective states of eachCPU 1906A comprised on compute sled 1904. In some embodiments, computestate 1984 may also include states of one or more memory resourcescomprised on compute sled 1904. The embodiments are not limited in thiscontext.

In various embodiments, robot 1960 may perform an automated computestate storage/transfer procedure in order to preserve the compute stateof compute sled 1904 during upkeep/repair of compute sled 1904. In somesuch embodiments, once compute state 1984 is stored on memory card 1918,robot 1960 may remove memory card 1918 from connector 1906B, performupkeep/repair of compute sled 1904, reinsert memory card 1918 intoconnector 1906B, and then restore compute sled 1904 to the compute state1984 stored on memory card 1918. For instance, in an example embodiment,robot 1960 may remove a CPU 1906A from a socket on compute sled 1904 andinsert a replacement CPU into that socket, and then cause compute sled1904 to be restored to the compute state 1984 stored on memory card1918. In various other embodiments, robot 1960 may perform an automatedcompute state storage/transfer procedure in order to replace computesled 1904 with another compute sled. In some such embodiments, oncecompute state 1984 is stored on memory card 1918, robot 1960 may removememory card 1918 from connector 1906B, insert memory card 1918 into aconnector on a replacement compute sled, insert the replacement computesled into a sled space of rack 1902 or another rack, and cause thereplacement compute sled to realize the compute state 1984 stored onmemory card 1918. The embodiments are not limited in this context.

FIG. 20 illustrates an example of an operating environment 2000.According to various embodiments, operating environment 2000 may berepresentative of the implementation of an automated data centermaintenance scheme according to which some aspects of automatedmaintenance operations involve collaboration/cooperation between robots.In operating environment 2000, in conjunction with performing acollaborative maintenance task, robots 2060A and 2060B may coordinatewith each other by exchanging interdevice coordination information 2086Aand 2086B via one or more communication links 2085. Communication links2085 may comprise wireless communication links, wired communicationlinks, or a combination of both. According to some embodiments, robots2060A and 2060B may be representative of robots 1360 that performoperations associated with automated data center maintenance in datacenter 1300 of FIGS. 13, 15, and 16. In various embodiments, one or bothof robots 2060A and 2060B may be implemented using automated maintenancedevice 1400 of FIG. 14.

It is worthy of note that the absence of automation coordinator 1555 inFIG. 20 is not intended to indicate that no aspects of automatedmaintenance would/could be centrally coordinated in operatingenvironment 2000. It is both possible and contemplated that in variousembodiments, distributed coordination may be implemented for someaspects of automated maintenance in a data center in which other aspectsof automated maintenance are centrally coordinated by an entity such asautomation coordinator 1555. For example, in operating environment 2000,a central automation coordinator may determine the need for performanceof the collaborative maintenance task, select robots 2060A and 2060B asthe robots that are to perform the collaborative maintenance task, andsend automation commands to cause robots 2060A and 2060B to initiate thecollaborative maintenance task. Robots 2060A and 2060B may thencoordinate directly with each other in conjunction with performing thephysical actions necessary to complete the collaborative maintenancetask. The embodiments are not limited to this example.

FIG. 21 illustrates an example of a logic flow 2100 that may berepresentative of the implementation of one or more of the disclosedtechniques according to some embodiments. For example, logic flow 2100may be representative of operations that automation coordinator 1555 mayperform in any of operating environments 1500, 1600, 1700, 1800, 1900,and 2000 of FIGS. 15-20 according to various embodiments. As shown inFIG. 21, at 2102, a maintenance task that is to be performed in a datacenter may be identified. For example, in operating environment 1500 ofFIG. 15, automation coordinator 1555 may identify a maintenance taskthat is to be performed in data center 1300.

At 2104, a determination may be made to initiate automated performanceof the maintenance task. For example, having added an identifiedmaintenance task to pending task queue 1578 in operating environment1500 of FIG. 15, automation coordinator 1555 may determine at asubsequent point in time that that maintenance task constitutes thehighest priority task in the pending task queue 1578 and thus that itsperformance should be initiated. In another example, rather than addingthe identified maintenance task to pending task queue 1578, automationcoordinator 1555 may determine to initiate performance of themaintenance task immediately after it is identified.

At 2106, an automated maintenance device to which to assign themaintenance task may be selected. For example, among one or more robots1360 comprised in candidate device pool 1580 in operating environment1500 of FIG. 15, automation coordinator 1555 may select a robot 1360 towhich to assign an identified maintenance task. It is worthy of notethat in some embodiments, the identified maintenance task may be handledby multiple robots according to a collaborate maintenance procedure. Insuch cases, more than one automated maintenance device may be selectedat 2106 as an assignee of the maintenance task. For example, inoperating environment 1500 of FIG. 15, automation coordinator 1555 mayselect multiple robots 1360 among those comprised in candidate devicepool 1580 that are to work together according to a collaborativemaintenance procedure to complete a maintenance task.

At 2108, one or more automation commands may be sent to cause anautomated maintenance device selected at 2106 to perform an automatedmaintenance procedure associated with the maintenance task. For example,in operating environment 1500 of FIG. 15, automation coordinator 1555may send one or more automation commands 1573 to cause a robot 1360 toperform an automated maintenance procedure associated with a maintenancetask to which that robot 1360 has been allocated. In some embodiments inwhich multiple automated maintenance devices are selected at 2106 asassignees of the same maintenance task, automation commands may be sentto multiple automated maintenance devices at 2108. For example, inoperating environment 1500 of FIG. 15, automation coordinator 1555 maysend respective automation command(s) 1573 to multiple robots 1360 tocause those robots to perform a collaborative maintenance procedureassociated with the maintenance task to be completed. The embodimentsare not limited to these examples.

FIG. 22 illustrates an example of a logic flow 2200 that may berepresentative of the implementation of one or more of the disclosedtechniques according to some embodiments. For example, logic flow 2200may be representative of operations that may be performed in variousembodiments by a robot such as a robot 1360 in one or both of operatingenvironments 1500 and 1600 of FIGS. 15 and 16 and/or any of robots 1760,1860, 1960, 2060A, and 2060B in operating environments 1700, 1800, 1900,and 2000 of FIGS. 17-20. As shown in FIG. 22, one or more automationcommands may be received from an automation coordinator of a data centerat 2202. For example, in operating environment 1500 of FIG. 15, a robot1360 may receive one or more automation commands 1573 from automationcoordinator 1555.

At 2204, an automated maintenance procedure may be identified based onthe one or more automation commands received at 2202. For example, basedon one or more automation commands 1573 received from automationcoordinator 1555 in operating environment 1500 of FIG. 15, a robot 1360may identify an automated maintenance procedure that it is to perform.The automated maintenance procedure identified at 2204 may then beperformed at 2206. In various embodiments, the identification of theautomated maintenance procedure at 2204 may be based on a maintenancetask code that is comprised in at least one of the received automationcommands, and is defined to correspond to a particular automatedmaintenance procedure. For example, based on a maintenance task codecomprised in an automation command 1573 received from automationcoordinator 1555, a robot 1360 in operating environment 1500 of FIG. 15may identify an automated DIMM testing procedure as an automatedmaintenance procedure to be performed. In various embodiments, the oneor more automation commands received at 2202 may collectively containone or more maintenance task parameters specifying particular details ofthe automated maintenance task, and such details may also be identifiedat 2204. For instance, in the context of the preceding example, therobot 1360 may identify—based on maintenance task parameters comprisedin one or more automation commands 1573 received from automationcoordinator 1555—details such as a physical resource ID of a DIMM to betested, an identity and location of a sled on which that DIMM resides,and an identity of a particular DIMM slot on that sled that currentlyhouses the DIMM. The embodiments are not limited to these examples.

FIG. 23 illustrates an example of a logic flow 2300 that may berepresentative of the implementation of one or more of the disclosedtechniques according to some embodiments. For example, logic flow 2300may be representative of operations that may be performed by robot 2060Aor robot 2060B in operating environment 2000 of FIG. 20. As shown inFIG. 23, a collaborative maintenance procedure that is to be performedin a data center may be identified at an automated maintenance device at2302. For example, in operating environment 2000 of FIG. 20, robot 2060Amay determine that a collaborative CPU replacement procedure is to beperformed. In some embodiments, the identification of the collaborativemaintenance procedure at 2302 may be based on one or more automationcommands received by the automated maintenance device from a centralizedautomation coordinator such as automation coordinator 1555. In variousother embodiments, the identification of the collaborative maintenanceprocedure at 2302 may be performed autonomously. For example, inoperating environment 1500 of FIG. 15, a robot 1360 may determine basedon analysis of telemetry data 1571 that a particular CPU ismalfunctioning, and may then identify a collaborative maintenanceprocedure to be performed in order to replace that malfunctioning CPU.The embodiments are not limited to this example.

A second automated maintenance device with which to collaborate duringperformance of the collaborative maintenance procedure may be identifiedat 2304, and interdevice coordination information may be sent to thesecond automated maintenance device at 2306 in order to initiate thecollaborative maintenance procedure. For example, in operatingenvironment 2000 of FIG. 20, robot 2060A may determine that it is tocollaborate with robot 2060B in conjunction with a collaborative CPUreplacement procedure, and may send interdevice coordination information2086A to robot 2086B in order to initiate that collaborative CPUreplacement procedure. In some embodiments, the identification of thesecond automated maintenance device may be based on information receivedfrom a centralized automation coordinator such as automation coordinator1555. For example, in some embodiments, a centralized automationcoordinator may be responsible for selecting the particular robots thatare to work together to perform the collaborative maintenance procedure,and the identity of the second automated maintenance device may beindicated by a parameter comprised in an automation command receivedfrom the centralized automation coordinator. In other embodiments, theidentification performed at 2304 may correspond to an autonomousselection of the second automated maintenance device. For example, inoperating environment 1500 of FIG. 15, a first robot 1360 may select asecond robot 1360 that is comprised among those in candidate device pool1580 as the second automated maintenance device that is to participatein the collaborative maintenance procedure. The embodiments are notlimited to these examples.

FIG. 24A illustrates an embodiment of a storage medium 2400. Storagemedium 2400 may comprise any computer-readable storage medium ormachine-readable storage medium, such as an optical, magnetic orsemiconductor storage medium. In some embodiments, storage medium 2400may comprise a non-transitory storage medium. In various embodiments,storage medium 2400 may comprise an article of manufacture. In someembodiments, storage medium 2400 may store computer-executableinstructions, such as computer-executable instructions to implementlogic flow 2100 of FIG. 21. Examples of a computer-readable storagemedium or machine-readable storage medium may include any tangible mediacapable of storing electronic data, including volatile memory ornon-volatile memory, removable or non-removable memory, erasable ornon-erasable memory, writeable or re-writeable memory, and so forth.Examples of computer-executable instructions may include any suitabletype of code, such as source code, compiled code, interpreted code,executable code, static code, dynamic code, object-oriented code, visualcode, and the like. The embodiments are not limited to these examples.

FIG. 24B illustrates an embodiment of a storage medium 2450. Storagemedium 2450 may comprise any computer-readable storage medium ormachine-readable storage medium, such as an optical, magnetic orsemiconductor storage medium. In some embodiments, storage medium 2450may comprise a non-transitory storage medium. In various embodiments,storage medium 2450 may comprise an article of manufacture. According tosome embodiments, storage medium 2450 may be representative of amemory/storage element 1467 comprised in automated maintenance device1400 of FIG. 14. In some embodiments, storage medium 2450 may storecomputer-executable instructions, such as computer-executableinstructions to implement one or both of logic flow 2200 of FIG. 22 andlogic flow 2300 of FIG. 23. Examples of a computer-readable storagemedium or machine-readable storage medium and of computer-executableinstructions may include any of the respective examples identified abovein reference to storage medium 2400 of FIG. 24A. The embodiments are notlimited to these examples.

FIG. 25 illustrates an embodiment of an exemplary computing architecture2500 that may be suitable for implementing various embodiments aspreviously described. In various embodiments, the computing architecture2500 may comprise or be implemented as part of an electronic device. Insome embodiments, the computing architecture 2500 may be representative,for example, of a computing device suitable for use in conjunction withimplementation of one or more of robots 1360, 1760, 1860, 1960, 2060A,and 2060B, automated maintenance device 1400, automation coordinator1555, and logic flows 2100, 2200, and 2300. The embodiments are notlimited in this context.

As used in this application, the terms “system” and “component” and“module” are intended to refer to a computer-related entity, eitherhardware, a combination of hardware and software, software, or softwarein execution, examples of which are provided by the exemplary computingarchitecture 2500. For example, a component can be, but is not limitedto being, a process running on a processor, a processor, a hard diskdrive, multiple storage drives (of optical and/or magnetic storagemedium), an object, an executable, a thread of execution, a program,and/or a computer. By way of illustration, both an application runningon a server and the server can be a component. One or more componentscan reside within a process and/or thread of execution, and a componentcan be localized on one computer and/or distributed between two or morecomputers. Further, components may be communicatively coupled to eachother by various types of communications media to coordinate operations.The coordination may involve the uni-directional or bi-directionalexchange of information. For instance, the components may communicateinformation in the form of signals communicated over the communicationsmedia. The information can be implemented as signals allocated tovarious signal lines. In such allocations, each message may be a signal.Further embodiments, however, may alternatively employ data messages.Such data messages may be sent across various connections. Exemplaryconnections include parallel interfaces, serial interfaces, and businterfaces.

The computing architecture 2500 includes various common computingelements, such as one or more processors, multi-core processors,co-processors, memory units, chipsets, controllers, peripherals,interfaces, oscillators, timing devices, video cards, audio cards,multimedia input/output (I/O) components, power supplies, and so forth.The embodiments, however, are not limited to implementation by thecomputing architecture 2500.

As shown in FIG. 25, according to computing architecture 2500, acomputer 2502 comprises a processing unit 2504, a system memory 2506 anda system bus 2508. In some embodiments, computer 2502 may comprise aserver. In some embodiments, computer 2502 may comprise a client. Theprocessing unit 2504 can be any of various commercially availableprocessors, including without limitation an AMD® Athlon®, Duron® andOpteron® processors; ARM® application, embedded and secure processors;IBM® and Motorola® DragonBall® and PowerPC® processors; IBM and Sony®Cell processors; Intel® Celeron®, Core (2) Duo®, Itanium®, Pentium®,Xeon®, and XScale® processors; and similar processors. Dualmicroprocessors, multi-core processors, and other multi processorarchitectures may also be employed as the processing unit 2504.

The system bus 2508 provides an interface for system componentsincluding, but not limited to, the system memory 2506 to the processingunit 2504. The system bus 2508 can be any of several types of busstructure that may further interconnect to a memory bus (with or withouta memory controller), a peripheral bus, and a local bus using any of avariety of commercially available bus architectures. Interface adaptersmay connect to the system bus 2508 via a slot architecture. Example slotarchitectures may include without limitation Accelerated Graphics Port(AGP), Card Bus, (Extended) Industry Standard Architecture ((E)ISA),Micro Channel Architecture (MCA), NuBus, Peripheral ComponentInterconnect (Extended) (PCI(X)), PCI Express, Personal Computer MemoryCard International Association (PCMCIA), and the like.

The system memory 2506 may include various types of computer-readablestorage media in the form of one or more higher speed memory units, suchas read-only memory (ROM), random-access memory (RAM), dynamic RAM(DRAM), Double-Data-Rate DRAM (DDRAM), synchronous DRAM (SDRAM), staticRAM (SRAM), programmable ROM (PROM), erasable programmable ROM (EPROM),electrically erasable programmable ROM (EEPROM), flash memory, polymermemory such as ferroelectric polymer memory, ovonic memory, phase changeor ferroelectric memory, silicon-oxide-nitride-oxide-silicon (SONOS)memory, magnetic or optical cards, an array of devices such as RedundantArray of Independent Disks (RAID) drives, solid state memory devices(e.g., USB memory, solid state drives (SSD) and any other type ofstorage media suitable for storing information. In the illustratedembodiment shown in FIG. 25, the system memory 2506 can includenon-volatile memory 2510 and/or volatile memory 2512. A basicinput/output system (BIOS) can be stored in the non-volatile memory2510.

The computer 2502 may include various types of computer-readable storagemedia in the form of one or more lower speed memory units, including aninternal (or external) hard disk drive (HDD) 2514, a magnetic floppydisk drive (FDD) 2516 to read from or write to a removable magnetic disk2518, and an optical disk drive 2520 to read from or write to aremovable optical disk 2522 (e.g., a CD-ROM or DVD). The HDD 2514, FDD2516 and optical disk drive 2520 can be connected to the system bus 2508by a HDD interface 2524, an FDD interface 2526 and an optical driveinterface 2528, respectively. The HDD interface 2524 for external driveimplementations can include at least one or both of Universal Serial Bus(USB) and IEEE 1394 interface technologies.

The drives and associated computer-readable media provide volatileand/or nonvolatile storage of data, data structures, computer-executableinstructions, and so forth. For example, a number of program modules canbe stored in the drives and memory units 2510, 2512, including anoperating system 2530, one or more application programs 2532, otherprogram modules 2534, and program data 2536.

A user can enter commands and information into the computer 2502 throughone or more wire/wireless input devices, for example, a keyboard 2538and a pointing device, such as a mouse 2540. Other input devices mayinclude microphones, infra-red (IR) remote controls, radio-frequency(RF) remote controls, game pads, stylus pens, card readers, dongles,finger print readers, gloves, graphics tablets, joysticks, keyboards,retina readers, touch screens (e.g., capacitive, resistive, etc.),trackballs, trackpads, sensors, styluses, and the like. These and otherinput devices are often connected to the processing unit 2504 through aninput device interface 2542 that is coupled to the system bus 2508, butcan be connected by other interfaces such as a parallel port, IEEE 1394serial port, a game port, a USB port, an IR interface, and so forth.

A monitor 2544 or other type of display device may also be connected tothe system bus 2508 via an interface, such as a video adaptor 2546. Themonitor 2544 may be internal or external to the computer 2502. Inaddition to the monitor 2544, a computer typically includes otherperipheral output devices, such as speakers, printers, and so forth.

The computer 2502 may operate in a networked environment using logicalconnections via wire and/or wireless communications to one or moreremote computers, such as a remote computer 2548. The remote computer2548 can be a workstation, a server computer, a router, a personalcomputer, portable computer, microprocessor-based entertainmentappliance, a peer device or other common network node, and typicallyincludes many or all of the elements described relative to the computer2502, although, for purposes of brevity, only a memory/storage device2550 is illustrated. The logical connections depicted includewire/wireless connectivity to a local area network (LAN) 2552 and/orlarger networks, for example, a wide area network (WAN) 2554. Such LANand WAN networking environments are commonplace in offices andcompanies, and facilitate enterprise-wide computer networks, such asintranets, all of which may connect to a global communications network,for example, the Internet.

When used in a LAN networking environment, the computer 2502 may beconnected to the LAN 2552 through a wire and/or wireless communicationnetwork interface or adaptor 2556. The adaptor 2556 can facilitate wireand/or wireless communications to the LAN 2552, which may also include awireless access point disposed thereon for communicating with thewireless functionality of the adaptor 2556.

When used in a WAN networking environment, the computer 2502 can includea modem 2558, or may be connected to a communications server on the WAN2554, or has other means for establishing communications over the WAN2554, such as by way of the Internet. The modem 2558, which can beinternal or external and a wire and/or wireless device, connects to thesystem bus 2508 via the input device interface 2542. In a networkedenvironment, program modules depicted relative to the computer 2502, orportions thereof, can be stored in the remote memory/storage device2550. It will be appreciated that the network connections shown areexemplary and other means of establishing a communications link betweenthe computers can be used.

The computer 2502 may be operable to communicate with wire and wirelessdevices or entities using the IEEE 802 family of standards, such aswireless devices operatively disposed in wireless communication (e.g.,IEEE 802.16 over-the-air modulation techniques). This includes at leastWi-Fi (or Wireless Fidelity), WiMax, and Bluetooth™ wirelesstechnologies, among others. Thus, the communication can be a predefinedstructure as with a conventional network or simply an ad hoccommunication between at least two devices. Wi-Fi networks use radiotechnologies called IEEE 802.11x (a, b, g, n, etc.) to provide secure,reliable, fast wireless connectivity. A Wi-Fi network can be used toconnect computers to each other, to the Internet, and to wire networks(which use IEEE 802.3-related media and functions).

FIG. 26 illustrates a block diagram of an exemplary communicationsarchitecture 2600 suitable for implementing various embodiments aspreviously described. The communications architecture 2600 includesvarious common communications elements, such as a transmitter, receiver,transceiver, radio, network interface, baseband processor, antenna,amplifiers, filters, power supplies, and so forth. The embodiments,however, are not limited to implementation by the communicationsarchitecture 2600.

As shown in FIG. 26, the communications architecture 2600 comprisesincludes one or more clients 2602 and servers 2604. The clients 2602 andthe servers 2604 are operatively connected to one or more respectiveclient data stores 2608 and server data stores 2610 that can be employedto store information local to the respective clients 2602 and servers2604, such as cookies and/or associated contextual information. Any oneof clients 2602 and/or servers 2604 may implement one or more of robots1360, 1760, 1860, 1960, 2060A, and 2060B, automated maintenance device1400, automation coordinator 1555, logic flows 2100, 2200, and 2300, andcomputing architecture 2500.

The clients 2602 and the servers 2604 may communicate informationbetween each other using a communication framework 2606. Thecommunications framework 2606 may implement any well-knowncommunications techniques and protocols. The communications framework2606 may be implemented as a packet-switched network (e.g., publicnetworks such as the Internet, private networks such as an enterpriseintranet, and so forth), a circuit-switched network (e.g., the publicswitched telephone network), or a combination of a packet-switchednetwork and a circuit-switched network (with suitable gateways andtranslators).

The communications framework 2606 may implement various networkinterfaces arranged to accept, communicate, and connect to acommunications network. A network interface may be regarded as aspecialized form of an input output interface. Network interfaces mayemploy connection protocols including without limitation direct connect,Ethernet (e.g., thick, thin, twisted pair 10/100/1000 Base T, and thelike), token ring, wireless network interfaces, cellular networkinterfaces, IEEE 802.11a-x network interfaces, IEEE 802.16 networkinterfaces, IEEE 802.20 network interfaces, and the like. Further,multiple network interfaces may be used to engage with variouscommunications network types. For example, multiple network interfacesmay be employed to allow for the communication over broadcast,multicast, and unicast networks. Should processing requirements dictatea greater amount speed and capacity, distributed network controllerarchitectures may similarly be employed to pool, load balance, andotherwise increase the communicative bandwidth required by clients 2602and the servers 2604. A communications network may be any one and thecombination of wired and/or wireless networks including withoutlimitation a direct interconnection, a secured custom connection, aprivate network (e.g., an enterprise intranet), a public network (e.g.,the Internet), a Personal Area Network (PAN), a Local Area Network(LAN), a Metropolitan Area Network (MAN), an Operating Missions as Nodeson the Internet (OMNI), a Wide Area Network (WAN), a wireless network, acellular network, and other communications networks.

As used herein, the term “circuitry” may refer to, be part of, orinclude an Application Specific Integrated Circuit (ASIC), an electroniccircuit, a processor (shared, dedicated, or group), and/or memory(shared, dedicated, or group) that execute one or more software orfirmware programs, a combinational logic circuit, and/or other suitablehardware components that provide the described functionality. In someembodiments, the circuitry may be implemented in, or functionsassociated with the circuitry may be implemented by, one or moresoftware or firmware modules. In some embodiments, circuitry may includelogic, at least partially operable in hardware. Embodiments describedherein may be implemented into a system using any suitably configuredhardware and/or software.

FIG. 27 illustrates an embodiment of a communication device 2700 thatmay implement one or more of robots 1360, 1760, 1860, 1960, 2060A, and2060B, automated maintenance device 1400, automation coordinator 1555,logic flows 2100, 2200, and 2300, storage media 2400 and 2450, computingarchitecture 2500, clients 2602, and servers 2604. In variousembodiments, device 2700 may comprise a logic circuit 2728. The logiccircuit 2728 may include physical circuits to perform operationsdescribed for one or more of robots 1360, 1760, 1860, 1960, 2060A, and2060B, automated maintenance device 1400, automation coordinator 1555,logic flows 2100, 2200, and 2300, computing architecture 2500, clients2602, and servers 2604 for example. As shown in FIG. 27, device 2700 mayinclude a radio interface 2710, baseband circuitry 2720, and computingplatform 2730, although the embodiments are not limited to thisconfiguration.

The device 2700 may implement some or all of the structure and/oroperations for one or more of robots 1360, 1760, 1860, 1960, 2060A, and2060B, automated maintenance device 1400, automation coordinator 1555,logic flows 2100, 2200, and 2300, storage media 2400 and 2450, computingarchitecture 2500, clients 2602, servers 2604, and logic circuit 2728 ina single computing entity, such as entirely within a single device.Alternatively, the device 2700 may distribute portions of the structureand/or operations for one or more of robots 1360, 1760, 1860, 1960,2060A, and 2060B, automated maintenance device 1400, automationcoordinator 1555, logic flows 2100, 2200, and 2300, storage media 2400and 2450, computing architecture 2500, clients 2602, servers 2604, andlogic circuit 2728 across multiple computing entities using adistributed system architecture, such as a client-server architecture, a3-tier architecture, an N-tier architecture, a tightly-coupled orclustered architecture, a peer-to-peer architecture, a master-slavearchitecture, a shared database architecture, and other types ofdistributed systems. The embodiments are not limited in this context.

In one embodiment, radio interface 2710 may include a component orcombination of components adapted for transmitting and/or receivingsingle-carrier or multi-carrier modulated signals (e.g., includingcomplementary code keying (CCK), orthogonal frequency divisionmultiplexing (OFDM), and/or single-carrier frequency division multipleaccess (SC-FDMA) symbols) although the embodiments are not limited toany specific over-the-air interface or modulation scheme. Radiointerface 2710 may include, for example, a receiver 2712, a frequencysynthesizer 2714, and/or a transmitter 2716. Radio interface 2710 mayinclude bias controls, a crystal oscillator and/or one or more antennas2718-f. In another embodiment, radio interface 2710 may use externalvoltage-controlled oscillators (VCOs), surface acoustic wave filters,intermediate frequency (IF) filters and/or RF filters, as desired. Dueto the variety of potential RF interface designs an expansivedescription thereof is omitted.

Baseband circuitry 2720 may communicate with radio interface 2710 toprocess receive and/or transmit signals and may include, for example, amixer for down-converting received RF signals, an analog-to-digitalconverter 2722 for converting analog signals to digital form, adigital-to-analog converter 2724 for converting digital signals toanalog form, and a mixer for up-converting signals for transmission.Further, baseband circuitry 2720 may include a baseband or physicallayer (PHY) processing circuit 2726 for PHY link layer processing ofrespective receive/transmit signals. Baseband circuitry 2720 mayinclude, for example, a medium access control (MAC) processing circuit2727 for MAC/data link layer processing. Baseband circuitry 2720 mayinclude a memory controller 2732 for communicating with MAC processingcircuit 2727 and/or a computing platform 2730, for example, via one ormore interfaces 2734.

In some embodiments, PHY processing circuit 2726 may include a frameconstruction and/or detection module, in combination with additionalcircuitry such as a buffer memory, to construct and/or deconstructcommunication frames. Alternatively or in addition, MAC processingcircuit 2727 may share processing for certain of these functions orperform these processes independent of PHY processing circuit 2726. Insome embodiments, MAC and PHY processing may be integrated into a singlecircuit.

The computing platform 2730 may provide computing functionality for thedevice 2700. As shown, the computing platform 2730 may include aprocessing component 2740. In addition to, or alternatively of, thebaseband circuitry 2720, the device 2700 may execute processingoperations or logic for one or more of robots 1360, 1760, 1860, 1960,2060A, and 2060B, automated maintenance device 1400, automationcoordinator 1555, logic flows 2100, 2200, and 2300, storage media 2400and 2450, computing architecture 2500, clients 2602, servers 2604, andlogic circuit 2728 using the processing component 2740. The processingcomponent 2740 (and/or PHY 2726 and/or MAC 2727) may comprise varioushardware elements, software elements, or a combination of both. Examplesof hardware elements may include devices, logic devices, components,processors, microprocessors, circuits, processor circuits, circuitelements (e.g., transistors, resistors, capacitors, inductors, and soforth), integrated circuits, application specific integrated circuits(ASIC), programmable logic devices (PLD), digital signal processors(DSP), field programmable gate array (FPGA), memory units, logic gates,registers, semiconductor device, chips, microchips, chip sets, and soforth. Examples of software elements may include software components,programs, applications, computer programs, application programs, systemprograms, software development programs, machine programs, operatingsystem software, middleware, firmware, software modules, routines,subroutines, functions, methods, procedures, software interfaces,application program interfaces (API), instruction sets, computing code,computer code, code segments, computer code segments, words, values,symbols, or any combination thereof. Determining whether an embodimentis implemented using hardware elements and/or software elements may varyin accordance with any number of factors, such as desired computationalrate, power levels, heat tolerances, processing cycle budget, input datarates, output data rates, memory resources, data bus speeds and otherdesign or performance constraints, as desired for a givenimplementation.

The computing platform 2730 may further include other platformcomponents 2750. Other platform components 2750 include common computingelements, such as one or more processors, multi-core processors,co-processors, memory units, chipsets, controllers, peripherals,interfaces, oscillators, timing devices, video cards, audio cards,multimedia input/output (I/O) components (e.g., digital displays), powersupplies, and so forth. Examples of memory units may include withoutlimitation various types of computer readable and machine readablestorage media in the form of one or more higher speed memory units, suchas read-only memory (ROM), random-access memory (RAM), dynamic RAM(DRAM), Double-Data-Rate DRAM (DDRAM), synchronous DRAM (SDRAM), staticRAM (SRAM), programmable ROM (PROM), erasable programmable ROM (EPROM),electrically erasable programmable ROM (EEPROM), flash memory, polymermemory such as ferroelectric polymer memory, ovonic memory, phase changeor ferroelectric memory, silicon-oxide-nitride-oxide-silicon (SONOS)memory, magnetic or optical cards, an array of devices such as RedundantArray of Independent Disks (RAID) drives, solid state memory devices(e.g., USB memory, solid state drives (SSD) and any other type ofstorage media suitable for storing information.

Device 2700 may be, for example, an ultra-mobile device, a mobiledevice, a fixed device, a machine-to-machine (M2M) device, a personaldigital assistant (PDA), a mobile computing device, a smart phone, atelephone, a digital telephone, a cellular telephone, user equipment,eBook readers, a handset, a one-way pager, a two-way pager, a messagingdevice, a computer, a personal computer (PC), a desktop computer, alaptop computer, a notebook computer, a netbook computer, a handheldcomputer, a tablet computer, a server, a server array or server farm, aweb server, a network server, an Internet server, a work station, amini-computer, a main frame computer, a supercomputer, a networkappliance, a web appliance, a distributed computing system,multiprocessor systems, processor-based systems, consumer electronics,programmable consumer electronics, game devices, display, television,digital television, set top box, wireless access point, base station,node B, subscriber station, mobile subscriber center, radio networkcontroller, router, hub, gateway, bridge, switch, machine, orcombination thereof. Accordingly, functions and/or specificconfigurations of device 2700 described herein, may be included oromitted in various embodiments of device 2700, as suitably desired.

Embodiments of device 2700 may be implemented using single input singleoutput (SISO) architectures. However, certain implementations mayinclude multiple antennas (e.g., antennas 2718-f) for transmissionand/or reception using adaptive antenna techniques for beamforming orspatial division multiple access (SDMA) and/or using MIMO communicationtechniques.

The components and features of device 2700 may be implemented using anycombination of discrete circuitry, application specific integratedcircuits (ASICs), logic gates and/or single chip architectures. Further,the features of device 2700 may be implemented using microcontrollers,programmable logic arrays and/or microprocessors or any combination ofthe foregoing where suitably appropriate. It is noted that hardware,firmware and/or software elements may be collectively or individuallyreferred to herein as “logic” or “circuit.”

It should be appreciated that the exemplary device 2700 shown in theblock diagram of FIG. 27 may represent one functionally descriptiveexample of many potential implementations. Accordingly, division,omission or inclusion of block functions depicted in the accompanyingfigures does not infer that the hardware components, circuits, softwareand/or elements for implementing these functions would be necessarily bedivided, omitted, or included in embodiments.

FIG. 28 illustrates an embodiment of a broadband wireless access system2800. As shown in FIG. 28, broadband wireless access system 2800 may bean internet protocol (IP) type network comprising an internet 2810 typenetwork or the like that is capable of supporting mobile wireless accessand/or fixed wireless access to internet 2810. In one or moreembodiments, broadband wireless access system 2800 may comprise any typeof orthogonal frequency division multiple access (OFDMA)-based orsingle-carrier frequency division multiple access (SC-FDMA)-basedwireless network, such as a system compliant with one or more of the3GPP LTE Specifications and/or IEEE 802.16 Standards, and the scope ofthe claimed subject matter is not limited in these respects.

In the exemplary broadband wireless access system 2800, radio accessnetworks (RANs) 2812 and 2818 are capable of coupling with evolved nodeBs (eNBs) 2814 and 2820, respectively, to provide wireless communicationbetween one or more fixed devices 2816 and internet 2810 and/or betweenor one or more mobile devices 2822 and Internet 2810. One example of afixed device 2816 and a mobile device 2822 is device 2700 of FIG. 27,with the fixed device 2816 comprising a stationary version of device2700 and the mobile device 2822 comprising a mobile version of device2700. RANs 2812 and 2818 may implement profiles that are capable ofdefining the mapping of network functions to one or more physicalentities on broadband wireless access system 2800. eNBs 2814 and 2820may comprise radio equipment to provide RF communication with fixeddevice 2816 and/or mobile device 2822, such as described with referenceto device 2700, and may comprise, for example, the PHY and MAC layerequipment in compliance with a 3GPP LTE Specification or an IEEE 802.16Standard. eNBs 2814 and 2820 may further comprise an IP backplane tocouple to Internet 2810 via RANs 2812 and 2818, respectively, althoughthe scope of the claimed subject matter is not limited in theserespects.

Broadband wireless access system 2800 may further comprise a visitedcore network (CN) 2824 and/or a home CN 2826, each of which may becapable of providing one or more network functions including but notlimited to proxy and/or relay type functions, for exampleauthentication, authorization and accounting (AAA) functions, dynamichost configuration protocol (DHCP) functions, or domain name servicecontrols or the like, domain gateways such as public switched telephonenetwork (PSTN) gateways or voice over internet protocol (VoIP) gateways,and/or internet protocol (IP) type server functions, or the like.However, these are merely example of the types of functions that arecapable of being provided by visited CN 2824 and/or home CN 2826, andthe scope of the claimed subject matter is not limited in theserespects. Visited CN 2824 may be referred to as a visited CN in the casewhere visited CN 2824 is not part of the regular service provider offixed device 2816 or mobile device 2822, for example where fixed device2816 or mobile device 2822 is roaming away from its respective home CN2826, or where broadband wireless access system 2800 is part of theregular service provider of fixed device 2816 or mobile device 2822 butwhere broadband wireless access system 2800 may be in another locationor state that is not the main or home location of fixed device 2816 ormobile device 2822. The embodiments are not limited in this context.

Fixed device 2816 may be located anywhere within range of one or both ofeNBs 2814 and 2820, such as in or near a home or business to providehome or business customer broadband access to Internet 2810 via eNBs2814 and 2820 and RANs 2812 and 2818, respectively, and home CN 2826. Itis worthy of note that although fixed device 2816 is generally disposedin a stationary location, it may be moved to different locations asneeded. Mobile device 2822 may be utilized at one or more locations ifmobile device 2822 is within range of one or both of eNBs 2814 and 2820,for example. In accordance with one or more embodiments, operationsupport system (OSS) 2828 may be part of broadband wireless accesssystem 2800 to provide management functions for broadband wirelessaccess system 2800 and to provide interfaces between functional entitiesof broadband wireless access system 2800. Broadband wireless accesssystem 2800 of FIG. 28 is merely one type of wireless network showing acertain number of the components of broadband wireless access system2800, and the scope of the claimed subject matter is not limited inthese respects.

FIG. 29 illustrates an embodiment of a wireless network 2900. As shownin FIG. 29, wireless network comprises an access point 2902 and wirelessstations 2904, 2906, and 2908. Any one of access point 2902 and wirelessstations 2904, 2906, and 2908 may potentially implement one or more ofrobots 1360, 1760, 1860, 1960, 2060A, and 2060B, automated maintenancedevice 1400, automation coordinator 1555, logic flows 2100, 2200, and2300, storage media 2400 and 2450, computing architecture 2500, clients2602, servers 2604, and communication device 2700.

In various embodiments, wireless network 2900 may comprise a wirelesslocal area network (WLAN), such as a WLAN implementing one or moreInstitute of Electrical and Electronics Engineers (IEEE) 802.11standards (sometimes collectively referred to as “Wi-Fi”). In some otherembodiments, wireless network 2900 may comprise another type of wirelessnetwork, and/or may implement other wireless communications standards.In various embodiments, for example, wireless network 2900 may comprisea WWAN or WPAN rather than a WLAN. The embodiments are not limited tothis example.

In some embodiments, wireless network 2900 may implement one or morebroadband wireless communications standards, such as 3G or 4G standards,including their revisions, progeny, and variants. Examples of 3G or 4Gwireless standards may include without limitation any of the IEEE802.16m and 802.16p standards, 3rd Generation Partnership Project (3GPP)Long Term Evolution (LTE) and LTE-Advanced (LTE-A) standards, andInternational Mobile Telecommunications Advanced (IMT-ADV) standards,including their revisions, progeny and variants. Other suitable examplesmay include, without limitation, Global System for Mobile Communications(GSM)/Enhanced Data Rates for GSM Evolution (EDGE) technologies,Universal Mobile Telecommunications System (UMTS)/High Speed PacketAccess (HSPA) technologies, Worldwide Interoperability for MicrowaveAccess (WiMAX) or the WiMAX II technologies, Code Division MultipleAccess (CDMA) 2000 system technologies (e.g., CDMA2000 1×RTT, CDMA2000EV-DO, CDMA EV-DV, and so forth), High Performance Radio MetropolitanArea Network (HIPERMAN) technologies as defined by the EuropeanTelecommunications Standards Institute (ETSI) Broadband Radio AccessNetworks (BRAN), Wireless Broadband (WiBro) technologies, GSM withGeneral Packet Radio Service (GPRS) system (GSM/GPRS) technologies, HighSpeed Downlink Packet Access (HSDPA) technologies, High Speed OrthogonalFrequency-Division Multiplexing (OFDM) Packet Access (HSOPA)technologies, High-Speed Uplink Packet Access (HSUPA) systemtechnologies, 3GPP Rel. 8-12 of LTE/System Architecture Evolution (SAE),and so forth. The embodiments are not limited in this context.

In various embodiments, wireless stations 2904, 2906, and 2908 maycommunicate with access point 2902 in order to obtain connectivity toone or more external data networks. In some embodiments, for example,wireless stations 2904, 2906, and 2908 may connect to the Internet 2912via access point 2902 and access network 2910. In various embodiments,access network 2910 may comprise a private network that providessubscription-based Internet-connectivity, such as an Internet ServiceProvider (ISP) network. The embodiments are not limited to this example.

In various embodiments, two or more of wireless stations 2904, 2906, and2908 may communicate with each other directly by exchanging peer-to-peercommunications. For example, in the example of FIG. 29, wirelessstations 2904 and 2906 communicate with each other directly byexchanging peer-to-peer communications 2914. In some embodiments, suchpeer-to-peer communications may be performed according to one or moreWi-Fi Alliance (WFA) standards. For example, in various embodiments,such peer-to-peer communications may be performed according to the WFAWi-Fi Direct standard, 2010 Release. In various embodiments, suchpeer-to-peer communications may additionally or alternatively beperformed using one or more interfaces, protocols, and/or standardsdeveloped by the WFA Wi-Fi Direct Services (WFDS) Task Group. Theembodiments are not limited to these examples.

Various embodiments may be implemented using hardware elements, softwareelements, or a combination of both. Examples of hardware elements mayinclude processors, microprocessors, circuits, circuit elements (e.g.,transistors, resistors, capacitors, inductors, and so forth), integratedcircuits, application specific integrated circuits (ASIC), programmablelogic devices (PLD), digital signal processors (DSP), field programmablegate array (FPGA), logic gates, registers, semiconductor device, chips,microchips, chip sets, and so forth. Examples of software may includesoftware components, programs, applications, computer programs,application programs, system programs, machine programs, operatingsystem software, middleware, firmware, software modules, routines,subroutines, functions, methods, procedures, software interfaces,application program interfaces (API), instruction sets, computing code,computer code, code segments, computer code segments, words, values,symbols, or any combination thereof. Determining whether an embodimentis implemented using hardware elements and/or software elements may varyin accordance with any number of factors, such as desired computationalrate, power levels, heat tolerances, processing cycle budget, input datarates, output data rates, memory resources, data bus speeds and otherdesign or performance constraints.

One or more aspects of at least one embodiment may be implemented byrepresentative instructions stored on a machine-readable medium whichrepresents various logic within the processor, which when read by amachine causes the machine to fabricate logic to perform the techniquesdescribed herein. Such representations, known as “IP cores” may bestored on a tangible, machine readable medium and supplied to variouscustomers or manufacturing facilities to load into the fabricationmachines that actually make the logic or processor. Some embodiments maybe implemented, for example, using a machine-readable medium or articlewhich may store an instruction or a set of instructions that, ifexecuted by a machine, may cause the machine to perform a method and/oroperations in accordance with the embodiments. Such a machine mayinclude, for example, any suitable processing platform, computingplatform, computing device, processing device, computing system,processing system, computer, processor, or the like, and may beimplemented using any suitable combination of hardware and/or software.The machine-readable medium or article may include, for example, anysuitable type of memory unit, memory device, memory article, memorymedium, storage device, storage article, storage medium and/or storageunit, for example, memory, removable or non-removable media, erasable ornon-erasable media, writeable or re-writeable media, digital or analogmedia, hard disk, floppy disk, Compact Disk Read Only Memory (CD-ROM),Compact Disk Recordable (CD-R), Compact Disk Rewriteable (CD-RW),optical disk, magnetic media, magneto-optical media, removable memorycards or disks, various types of Digital Versatile Disk (DVD), a tape, acassette, or the like. The instructions may include any suitable type ofcode, such as source code, compiled code, interpreted code, executablecode, static code, dynamic code, encrypted code, and the like,implemented using any suitable high-level, low-level, object-oriented,visual, compiled and/or interpreted programming language.

The following examples pertain to further embodiments:

Example 1 is a method for automated data center maintenance, comprisingprocessing, by processing circuitry of an automated maintenance device,an automation command received from an automation coordinator for a datacenter, identifying an automated maintenance procedure based on thereceived automation command, and performing the identified automatedmaintenance procedure.

Example 2 is the method of Example 1, the identified automatedmaintenance procedure to comprise a sled replacement procedure.

Example 3 is the method of Example 2, the sled replacement procedure tocomprise replacing a compute sled.

Example 4 is the method of Example 3, the sled replacement procedure tocomprise removing the compute sled from a sled space, removing a memorycard from a connector slot of the compute sled, inserting the memorycard into a connector slot of a replacement compute sled, and insertingthe replacement compute sled into the sled space.

Example 5 is the method of Example 4, the memory card to store a computestate of the compute sled.

Example 6 is the method of Example 5, the sled replacement procedure tocomprise initiating a restoration of the stored compute state on thereplacement compute sled.

Example 7 is the method of Example 2, the sled replacement procedure tocomprise replacing an accelerator sled.

Example 8 is the method of Example 2, the sled replacement procedure tocomprise replacing a memory sled.

Example 9 is the method of Example 2, the sled replacement procedure tocomprise replacing a storage sled.

Example 10 is the method of Example 1, the identified automatedmaintenance procedure to comprise a component replacement procedure.

Example 11 is the method of Example 10, the component replacementprocedure to comprise removing a component from a socket of a sled, andinserting a replacement component into the socket.

Example 12 is the method of Example 11, the component to comprise aprocessor.

Example 13 is the method of Example 11, the component to comprise afield-programmable gate array (FPGA).

Example 14 is the method of Example 11, the component to comprise amemory module.

Example 15 is the method of Example 11, the component to comprise anon-volatile storage device.

Example 16 is the method of Example 15, the non-volatile storage deviceto comprise a solid-state drive (SSD).

Example 17 is the method of Example 16, the SSD to comprise athree-dimensional (3D) NAND SSD.

Example 18 is the method of Example 10, the component replacementprocedure to comprise a cache memory replacement procedure.

Example 19 is the method of Example 18, the cache memory replacementprocedure to comprise replacing one or more cache memory modules of aprocessor on a sled.

Example 20 is the method of Example 19, the cache memory replacementprocedure to comprise removing a heat sink from atop the processor,removing the processor from a socket to facilitate access to one or morecache memory modules underlying the processor, removing the one or cachememory modules, inserting one or more replacement cache memory modules,reinserting the processor into the socket, and reinstalling the heatsink.

Example 21 is the method of Example 1, the identified automatedmaintenance procedure to comprise a component servicing procedure.

Example 22 is the method of Example 21, the component servicingprocedure to comprise servicing a component on a sled.

Example 23 is the method of Example 22, the component servicingprocedure to comprise removing the sled from a sled space of a rack.

Example 24 is the method of any of Examples 22 to 23, the componentservicing procedure to comprise removing the component from the sled.

Example 25 is the method of any of Examples 22 to 24, the componentservicing procedure to comprise testing the component.

Example 26 is the method of any of Examples 22 to 25, the componentservicing procedure to comprise cleaning the component.

Example 27 is the method of any of Examples 22 to 26, the componentservicing procedure to comprise power-cycling the component.

Example 28 is the method of any of Examples 22 to 27, the componentservicing procedure to comprise capturing one or more images of thecomponent.

Example 29 is the method of Example 28, comprising sending the one ormore captured images to the automation coordinator.

Example 30 is the method of any of Examples 22 to 29, the component tocomprise a processor.

Example 31 is the method of any of Examples 22 to 29, the component tocomprise a field-programmable gate array (FPGA).

Example 32 is the method of any of Examples 22 to 29, the component tocomprise a memory module.

Example 33 is the method of any of Examples 22 to 29, the component tocomprise a non-volatile storage device.

Example 34 is the method of Example 33, the non-volatile storage deviceto comprise a solid-state drive (SSD).

Example 35 is the method of Example 34, the SSD to comprise athree-dimensional (3D) NAND SSD.

Example 36 is the method of any of Examples 1 to 35, comprisingidentifying the automated maintenance procedure based on a maintenancetask code comprised in the received automation command.

Example 37 is the method of any of Examples 1 to 36, comprisingperforming the identified automated maintenance procedure based on oneor more maintenance task parameters.

Example 38 is the method of Example 37, the one or more maintenance taskparameters to be comprised in the received automation command.

Example 39 is the method of Example 37, at least one of the one or moremaintenance task parameters to be comprised in a second automationcommand received from the automation coordinator.

Example 40 is the method of any of Examples 37 to 39, the one or moremaintenance task parameters to include one or more location parameters.

Example 41 is the method of Example 40, the one or more locationparameters to include a rack identifier (ID) associated with a rackwithin the data center.

Example 42 is the method of any of Examples 40 to 41, the one or morelocation parameters to include a sled space identifier (ID) associatedwith a sled space within the data center.

Example 43 is the method of any of Examples 40 to 42, the one or morelocation parameters to include a slot identifier (ID) associated with aconnector socket on a sled within the data center.

Example 44 is the method of any of Examples 37 to 43, the one or moremaintenance task parameters to include a sled identifier (ID) associatedwith a sled within the data center.

Example 45 is the method of any of Examples 37 to 44, the one or moremaintenance task parameters to include a component identifier (ID)associated with a component on a sled within the data center.

Example 46 is the method of any of Examples 1 to 45, the automationcommand to be comprised in signals received via a communicationinterface of the automated maintenance device.

Example 47 is the method of Example 46, the communication interface tocomprise a radio frequency (RF) interface, the signals to comprise RFsignals.

Example 48 is the method of any of Examples 1 to 47, comprising sendinga message to the automation coordinator to acknowledge the receivedautomation command.

Example 49 is the method of any of Examples 1 to 48, comprising sendinga message to the automation coordinator to report a result of theautomated maintenance procedure.

Example 50 is the method of any of Examples 1 to 49, comprising sendingposition data to the automation coordinator, the position data toindicate a position of the automated maintenance device within the datacenter.

Example 51 is the method of any of Examples 1 to 50, comprising sendingassistance data to the automation coordinator, the assistance data tocomprise an image of a component that is to be manually replaced orserviced.

Example 52 is the method of any of Example 1 to 51, comprising sendingenvironmental data to the automation coordinator, the environmental datato comprise measurements of one or more aspects of ambient conditionswithin the data center.

Example 53 is the method of Example 52, comprising one or more sensorsto generate the measurements comprised in the environmental data.

Example 54 is the method of any of Examples 52 to 53, the environmentaldata to comprise one or more temperature measurements.

Example 55 is the method of any of Examples 52 to 54, the environmentaldata to comprise one or more humidity measurements.

Example 56 is the method of any of Examples 52 to 55, the environmentaldata to comprise one or more air quality measurements.

Example 57 is the method of any of Examples 52 to 56, the environmentaldata to comprise one or more pressure measurements.

Example 58 is a computer-readable storage medium storing instructionsthat, when executed, cause an automated maintenance device to perform amethod according to any of Examples 1 to 57.

Example 59 is an automated maintenance device, comprising processingcircuitry and computer-readable storage media storing instructions forexecution by the processing circuitry to cause the automated maintenancedevice to perform a method according to any of Examples 1 to 57.

Example 60 is a method for coordination of automated data centermaintenance, comprising identifying, by processing circuitry, amaintenance task to be performed in a data center, determining toinitiate automated performance of the maintenance task, selecting anautomated maintenance device to which to assign the maintenance task,and sending an automation command to cause the automated maintenancedevice to perform an automated maintenance procedure associated with themaintenance task.

Example 61 is the method of Example 60, comprising identifying themaintenance task based on telemetry data associated with one or morephysical resources of the data center.

Example 62 is the method of Example 61, comprising receiving thetelemetry data via a telemetry framework of the data center.

Example 63 is the method of any of Examples 61 to 62, the telemetry datato include one or more telemetry metrics associated with a physicalcompute resource.

Example 64 is the method of any of Examples 61 to 63, the telemetry datato include one or more telemetry metrics associated with a physicalaccelerator resource.

Example 65 is the method of any of Examples 61 to 64, the telemetry datato include one or more telemetry metrics associated with a physicalmemory resource.

Example 66 is the method of any of Examples 61 to 65, the telemetry datato include one or more telemetry metrics associated with a physicalstorage resource.

Example 67 is the method of any of Examples 60 to 66, comprisingidentifying the maintenance task based on environmental data receivedfrom one or more automated maintenance devices of the data center.

Example 68 is the method of Example 67, the environmental data toinclude one or more temperature measurements.

Example 69 is the method of any of Examples 67 to 68, the environmentaldata to include one or more humidity measurements.

Example 70 is the method of any of Examples 67 to 69, the environmentaldata to include one or more air quality measurements.

Example 71 is the method of any of Examples 67 to 70, the environmentaldata to include one or more pressure measurements.

Example 72 is the method of any of Examples 60 to 71, comprising addingthe maintenance task to a pending task queue following identification ofthe maintenance task.

Example 73 is the method of Example 72, comprising determining toinitiate automated performance of the maintenance task based on adetermination that the maintenance task constitutes a highest prioritytask among one or more maintenance tasks comprised in the pending taskqueue.

Example 74 is the method of any of Examples 60 to 73, comprisingselecting the automated maintenance device from among one or moreautomated maintenance devices in a candidate device pool.

Example 75 is the method of any of Examples 60 to 74, comprisingselecting the automated maintenance device based on one or morecapabilities of the automated maintenance device.

Example 76 is the method of any of Examples 60 to 75, comprisingselecting the automated maintenance device based on position datareceived from the automated maintenance device.

Example 77 is the method of any of Examples 60 to 76, the automationcommand to comprise a maintenance task code indicating a task typeassociated with the maintenance task.

Example 78 is the method of any of Examples 60 to 77, the automationcommand to comprise location information associated with the maintenancetask.

Example 79 is the method of Example 78, the location information toinclude a rack identifier (ID) associated with a rack within the datacenter.

Example 80 is the method of any of Examples 78 to 79, the locationinformation to include a sled space identifier (ID) associated with asled space within the data center.

Example 81 is the method of any of Examples 78 to 80, the locationinformation to include a slot identifier (ID) associated with aconnector socket on a sled within the data center.

Example 82 is the method of any of Examples 60 to 81, the automationcommand to comprise a sled identifier (ID) associated with a sled withinthe data center.

Example 83 is the method of any of Examples 60 to 82, the automationcommand to comprise a physical resource identifier (ID) associated witha physical resource within the data center.

Example 84 is the method of any of Examples 60 to 81, the maintenancetask to comprise replacement of a sled.

Example 85 is the method of Example 83, the sled to comprise a computesled, an accelerator sled, a memory sled, or a storage sled.

Example 86 is the method of any of Examples 60 to 81, the maintenancetask to comprise replacement of one or more components of a sled.

Example 87 is the method of any of Examples 60 to 81, the maintenancetask to comprise repair of one or more components of a sled.

Example 88 is the method of any of Examples 60 to 81, the maintenancetask to comprise testing of one or more components of a sled.

Example 89 is the method of any of Examples 60 to 81, the maintenancetask to comprise cleaning of one or more components of a sled.

Example 90 is the method of any of Examples 60 to 81, the maintenancetask to comprise power cycling one or more memory modules.

Example 91 is the method of any of Examples 60 to 81, the maintenancetask to comprise power cycling one or more non-volatile storage devices.

Example 92 is the method of any of Examples 60 to 81, the maintenancetask to comprise storing a compute state of a compute sled, replacingthe compute sled with a second compute sled, and transferring the storedcompute state to the second compute sled.

Example 93 is the method of any of Examples 60 to 81, the maintenancetask to comprise replacing one or more cache memory modules of aprocessor.

Example 94 is a computer-readable storage medium storing instructionsthat, when executed by an automation coordinator for a data center,cause the automation coordinator to perform a method according to any ofExamples 60 to 93.

Example 95 is an apparatus, comprising processing circuitry andcomputer-readable storage media storing instructions for execution bythe processing circuitry to perform a method according to any ofExamples 60 to 93.

Example 96 is a method for automated data center maintenance, comprisingidentifying, by processing circuitry of an automated maintenance device,a collaborative maintenance procedure to be performed in a data center,identifying a second automated maintenance device with which tocollaborate during performance of the collaborative maintenanceprocedure, and sending interdevice coordination information to thesecond automated maintenance device to initiate the collaborativemaintenance procedure.

Example 97 is the method of Example 96, comprising identifying thecollaborative maintenance procedure based on telemetry data associatedwith one or more physical resources of the data center.

Example 98 is the method of Example 97, the telemetry data to includeone or more telemetry metrics associated with a physical computeresource.

Example 99 is the method of any of Examples 97 to 98, the telemetry datato include one or more telemetry metrics associated with a physicalaccelerator resource.

Example 100 is the method of any of Examples 97 to 99, the telemetrydata to include one or more telemetry metrics associated with a physicalmemory resource.

Example 101 is the method of any of Examples 97 to 100, the telemetrydata to include one or more telemetry metrics associated with a physicalstorage resource.

Example 102 is the method of any of Examples 96 to 101, comprisingidentifying the collaborative maintenance procedure based onenvironmental data comprising measurements of one or more aspects ofambient conditions within the data center.

Example 103 is the method of Example 102, comprising one or more sensorsto generate the measurements comprised in the environmental data.

Example 104 is the method of any of Examples 102 to 103, theenvironmental data to comprise one or more temperature measurements.

Example 105 is the method of any of Examples 102 to 104, theenvironmental data to comprise one or more humidity measurements.

Example 106 is the method of any of Examples 102 to 105, theenvironmental data to comprise one or more air quality measurements.

Example 107 is the method of any of Examples 102 to 106, theenvironmental data to comprise one or more pressure measurements.

Example 108 is the method of Example 96, comprising identifying thecollaborative maintenance procedure based on an automation commandreceived from an automation coordinator for the data center.

Example 109 is the method of Example 108, comprising identifying thecollaborative maintenance procedure based on a maintenance task codecomprised in the received automation command.

Example 110 is the method of any of Examples 96 to 109, comprisingselecting the second automated maintenance device from among a pluralityof automated maintenance devices in a candidate device pool for the datacenter.

Example 111 is the method of any of Examples 96 to 110, comprisingidentifying the second automated maintenance device based on a parametercomprised in a command received from an automation coordinator for thedata center.

Example 112 is the method of any of Examples 96 to 111, thecollaborative maintenance procedure to comprise replacing a sled.

Example 113 is the method of Example 112, the sled to comprise a computesled.

Example 114 is the method of Example 113, the collaborative maintenanceprocedure to comprise removing the compute sled from a sled space,removing a memory card from a connector slot of the compute sled,inserting the memory card into a connector slot of a replacement computesled, and inserting the replacement compute sled into the sled space.

Example 115 is the method of Example 114, the memory card to store acompute state of the compute sled.

Example 116 is the method of Example 115, the collaborative maintenanceprocedure to comprise initiating a restoration of the stored computestate on the replacement compute sled.

Example 117 is the method of Example 112, the sled to comprise anaccelerator sled, a memory sled, or a storage sled.

Example 118 is the method of any of Examples 96 to 111, thecollaborative maintenance procedure to comprise replacing a component ona sled.

Example 119 is the method of Example 118, the component to comprise aprocessor.

Example 120 is the method of Example 118, the component to comprise afield-programmable gate array (FPGA).

Example 121 is the method of Example 118, the component to comprise amemory module.

Example 122 is the method of Example 118, the component to comprise anon-volatile storage device.

Example 123 is the method of Example 122, the non-volatile storagedevice to comprise a solid-state drive (SSD).

Example 124 is the method of Example 123, the SSD to comprise athree-dimensional (3D) NAND SSD.

Example 125 is the method of any of Examples 96 to 111, thecollaborative maintenance procedure to comprise replacing one or morecache memory modules of a processor on a sled.

Example 126 is the method of Example 125, the collaborative maintenanceprocedure to comprise removing a heat sink from atop the processor,removing the processor from a socket to facilitate access to one or morecache memory modules underlying the processor, removing the one or cachememory modules, inserting one or more replacement cache memory modules,reinserting the processor into the socket, and reinstalling the heatsink.

Example 127 is the method of any of Examples 96 to 111, thecollaborative maintenance procedure to comprise servicing a component ona sled.

Example 128 is the method of Example 127, the collaborative maintenanceprocedure to comprise removing the sled from a sled space of a rack.

Example 129 is the method of any of Examples 127 to 128, thecollaborative maintenance procedure to comprise removing the componentfrom the sled.

Example 130 is the method of any of Examples 127 to 129, thecollaborative maintenance procedure to comprise testing the component.

Example 131 is the method of any of Examples 127 to 130, thecollaborative maintenance procedure to comprise cleaning the component.

Example 132 is the method of any of Examples 127 to 131, thecollaborative maintenance procedure to comprise power-cycling thecomponent.

Example 133 is the method of any of Examples 127 to 132, thecollaborative maintenance procedure to comprise capturing one or moreimages of the component.

Example 134 is the method of any of Examples 127 to 133, the componentto comprise a processor.

Example 135 is the method of any of Examples 127 to 133, the componentto comprise a field-programmable gate array (FPGA).

Example 136 is the method of any of Examples 127 to 133, the componentto comprise a memory module.

Example 137 is the method of any of Examples 127 to 133, the componentto comprise a non-volatile storage device.

Example 138 is the method of Example 137, the non-volatile storagedevice to comprise a solid-state drive (SSD).

Example 139 is the method of Example 138, the SSD to comprise athree-dimensional (3D) NAND SSD.

Example 140 is the method of any of Examples 96 to 139, the interdevicecoordination information to comprise a rack identifier (ID) associatedwith a rack within the data center.

Example 141 is the method of any of Examples 96 to 140, the interdevicecoordination information to comprise a sled space identifier (ID)associated with a sled space within the data center.

Example 142 is the method of any of Examples 96 to 141, the interdevicecoordination information to comprise a slot identifier (ID) associatedwith a connector socket on a sled within the data center.

Example 143 is the method of any of Examples 96 to 142, the interdevicecoordination information to comprise a sled identifier (ID) associatedwith a sled within the data center.

Example 144 is the method of any of Examples 96 to 143, the interdevicecoordination information to comprise a component identifier (ID)associated with a component on a sled within the data center.

Example 145 is a computer-readable storage medium storing instructionsthat, when executed, cause an automated maintenance device to perform amethod according to any of Examples 96 to 144.

Example 146 is an automated maintenance device, comprising processingcircuitry and computer-readable storage media storing instructions forexecution by the processing circuitry to cause the automated maintenancedevice to perform a method according to any of Examples 96 to 144.

Example 147 is an automated maintenance device, comprising means forreceiving an automation command from an automation coordinator for adata center, means for identifying an automated maintenance procedurebased on the received automation command, and means for performing theidentified automated maintenance procedure.

Example 148 is the automated maintenance device of Example 147, theidentified automated maintenance procedure to comprise a sledreplacement procedure.

Example 149 is the automated maintenance device of Example 148, the sledreplacement procedure to comprise removing a compute sled from a sledspace, removing a memory card from a connector slot of the compute sled,inserting the memory card into a connector slot of a replacement computesled, and inserting the replacement compute sled into the sled space.

Example 150 is the automated maintenance device of Example 149, thememory card to store a compute state of the compute sled.

Example 151 is the automated maintenance device of Example 150, the sledreplacement procedure to comprise initiating a restoration of the storedcompute state on the replacement compute sled.

Example 152 is the automated maintenance device of Example 148, the sledreplacement procedure to comprise replacing an accelerator sled, amemory sled, or a storage sled.

Example 153 is the automated maintenance device of Example 147, theidentified automated maintenance procedure to comprise a componentreplacement procedure.

Example 154 is the automated maintenance device of Example 153, thecomponent replacement procedure to comprise removing a component from asocket of a sled, and inserting a replacement component into the socket.

Example 155 is the automated maintenance device of Example 154, thecomponent to comprise a processor, a field-programmable gate array(FPGA), a memory module, or a solid-state drive (SSD).

Example 156 is the automated maintenance device of Example 153, thecomponent replacement procedure to comprise a cache memory replacementprocedure.

Example 157 is the automated maintenance device of Example 156, thecache memory replacement procedure to comprise replacing one or morecache memory modules of a processor on a sled.

Example 158 is the automated maintenance device of Example 157, thecache memory replacement procedure to comprise removing a heat sink fromatop the processor, removing the processor from a socket to facilitateaccess to one or more cache memory modules underlying the processor,removing the one or cache memory modules, inserting one or morereplacement cache memory modules, reinserting the processor into thesocket, and reinstalling the heat sink.

Example 159 is the automated maintenance device of Example 147, theidentified automated maintenance procedure to comprise a componentservicing procedure.

Example 160 is the automated maintenance device of Example 159, thecomponent servicing procedure to comprise servicing a component on asled.

Example 161 is the automated maintenance device of Example 160, thecomponent servicing procedure to comprise removing the sled from a sledspace of a rack.

Example 162 is the automated maintenance device of any of Examples 160to 161, the component servicing procedure to comprise removing thecomponent from the sled.

Example 163 is the automated maintenance device of any of Examples 160to 162, the component servicing procedure to comprise testing thecomponent.

Example 164 is the automated maintenance device of any of Examples 160to 163, the component servicing procedure to comprise cleaning thecomponent.

Example 165 is the automated maintenance device of any of Examples 160to 164, the component servicing procedure to comprise power-cycling thecomponent.

Example 166 is the automated maintenance device of any of Examples 160to 165, the component servicing procedure to comprise capturing one ormore images of the component.

Example 167 is the automated maintenance device of any of Examples 160to 166, the component to comprise a processor, a field-programmable gatearray (FPGA), a memory module, or a solid-state drive (SSD).

Example 168 is the automated maintenance device of any of Examples 147to 167, comprising means for identifying the automated maintenanceprocedure based on a maintenance task code comprised in the receivedautomation command.

Example 169 is the automated maintenance device of any of Examples 147to 168, comprising means for performing the identified automatedmaintenance procedure based on one or more maintenance task parameters.

Example 170 is the automated maintenance device of Example 169, the oneor more maintenance task parameters to be comprised in the receivedautomation command.

Example 171 is the automated maintenance device of Example 169, at leastone of the one or more maintenance task parameters to be comprised in asecond automation command received from the automation coordinator.

Example 172 is the automated maintenance device of any of Examples 169to 171, the one or more maintenance task parameters to include one ormore location parameters.

Example 173 is the automated maintenance device of Example 172, the oneor more location parameters to include a rack identifier (ID) associatedwith a rack within the data center.

Example 174 is the automated maintenance device of any of Examples 172to 173, the one or more location parameters to include a sled spaceidentifier (ID) associated with a sled space within the data center.

Example 175 is the automated maintenance device of any of Examples 172to 174, the one or more location parameters to include a slot identifier(ID) associated with a connector socket on a sled within the datacenter.

Example 176 is the automated maintenance device of any of Examples 169to 175, the one or more maintenance task parameters to include a sledidentifier (ID) associated with a sled within the data center.

Example 177 is the automated maintenance device of any of Examples 169to 176, the one or more maintenance task parameters to include acomponent identifier (ID) associated with a component on a sled withinthe data center.

Example 178 is the automated maintenance device of any of Examples 147to 177, the automation command to be comprised in signals received via acommunication interface of the automated maintenance device.

Example 179 is the automated maintenance device of Example 178, thecommunication interface to comprise a radio frequency (RF) interface,the signals to comprise RF signals.

Example 180 is the automated maintenance device of any of Examples 147to 179, comprising means for sending a message to the automationcoordinator to acknowledge the received automation command.

Example 181 is the automated maintenance device of any of Examples 147to 180, comprising means for sending a message to the automationcoordinator to report a result of the automated maintenance procedure.

Example 182 is the automated maintenance device of any of Examples 147to 181, comprising means for sending position data to the automationcoordinator, the position data to indicate a position of the automatedmaintenance device within the data center.

Example 183 is the automated maintenance device of any of Examples 147to 182, comprising means for sending assistance data to the automationcoordinator, the assistance data to comprise an image of a componentthat is to be manually replaced or serviced.

Example 184 is the automated maintenance device of any of Example 147 to183, comprising means for sending environmental data to the automationcoordinator, the environmental data to comprise measurements of one ormore aspects of ambient conditions within the data center.

Example 185 is the automated maintenance device of Example 184,comprising means for generating the measurements comprised in theenvironmental data.

Example 186 is the automated maintenance device of any of Examples 184to 185, the environmental data to comprise one or more temperaturemeasurements.

Example 187 is the automated maintenance device of any of Examples 184to 186, the environmental data to comprise one or more humiditymeasurements.

Example 188 is the automated maintenance device of any of Examples 184to 187, the environmental data to comprise one or more air qualitymeasurements.

Example 189 is the automated maintenance device of any of Examples 184to 188, the environmental data to comprise one or more pressuremeasurements.

Example 189 is an apparatus for coordination of automated data centermaintenance, comprising means for identifying a maintenance task to beperformed in a data center, means for determining to initiate automatedperformance of the maintenance task, means for selecting an automatedmaintenance device to which to assign the maintenance task, and meansfor sending an automation command to cause the automated maintenancedevice to perform an automated maintenance procedure associated with themaintenance task.

Example 190 is the apparatus of Example 189, comprising means foridentifying the maintenance task based on telemetry data associated withone or more physical resources of the data center.

Example 191 is the apparatus of Example 190, comprising means forreceiving the telemetry data via a telemetry framework of the datacenter.

Example 192 is the apparatus of any of Examples 190 to 191, thetelemetry data to include one or more telemetry metrics associated witha physical compute resource.

Example 193 is the apparatus of any of Examples 190 to 192, thetelemetry data to include one or more telemetry metrics associated witha physical accelerator resource.

Example 194 is the apparatus of any of Examples 190 to 193, thetelemetry data to include one or more telemetry metrics associated witha physical memory resource.

Example 195 is the apparatus of any of Examples 190 to 194, thetelemetry data to include one or more telemetry metrics associated witha physical storage resource.

Example 196 is the apparatus of any of Examples 189 to 195, comprisingmeans for identifying the maintenance task based on environmental datareceived from one or more automated maintenance devices of the datacenter.

Example 197 is the apparatus of Example 196, the environmental data toinclude one or more temperature measurements.

Example 198 is the apparatus of any of Examples 196 to 197, theenvironmental data to include one or more humidity measurements.

Example 199 is the apparatus of any of Examples 196 to 198, theenvironmental data to include one or more air quality measurements.

Example 200 is the apparatus of any of Examples 196 to 199, theenvironmental data to include one or more pressure measurements.

Example 201 is the apparatus of any of Examples 189 to 200, comprisingmeans for adding the maintenance task to a pending task queue followingidentification of the maintenance task.

Example 202 is the apparatus of Example 201, comprising means fordetermining to initiate automated performance of the maintenance taskbased on a determination that the maintenance task constitutes a highestpriority task among one or more maintenance tasks comprised in thepending task queue.

Example 203 is the apparatus of any of Examples 189 to 202, comprisingmeans for selecting the automated maintenance device from among one ormore automated maintenance devices in a candidate device pool.

Example 204 is the apparatus of any of Examples 189 to 203, comprisingmeans for selecting the automated maintenance device based on one ormore capabilities of the automated maintenance device.

Example 205 is the apparatus of any of Examples 189 to 204, comprisingmeans for selecting the automated maintenance device based on positiondata received from the automated maintenance device.

Example 206 is the apparatus of any of Examples 189 to 205, theautomation command to comprise a maintenance task code indicating a tasktype associated with the maintenance task.

Example 207 is the apparatus of any of Examples 189 to 206, theautomation command to comprise location information associated with themaintenance task.

Example 208 is the apparatus of Example 207, the location information toinclude a rack identifier (ID) associated with a rack within the datacenter.

Example 209 is the apparatus of any of Examples 207 to 208, the locationinformation to include a sled space identifier (ID) associated with asled space within the data center.

Example 210 is the apparatus of any of Examples 207 to 209, the locationinformation to include a slot identifier (ID) associated with aconnector socket on a sled within the data center.

Example 211 is the apparatus of any of Examples 189 to 210, theautomation command to comprise a sled identifier (ID) associated with asled within the data center.

Example 212 is the apparatus of any of Examples 189 to 211, theautomation command to comprise a physical resource identifier (ID)associated with a physical resource within the data center.

Example 213 is the apparatus of any of Examples 189 to 212, themaintenance task to comprise replacement of a sled.

Example 214 is the apparatus of Example 213, the sled to comprise acompute sled, an accelerator sled, a memory sled, or a storage sled.

Example 215 is the apparatus of any of Examples 189 to 212, themaintenance task to comprise replacement of one or more components of asled.

Example 216 is the apparatus of any of Examples 189 to 212, themaintenance task to comprise repair of one or more components of a sled.

Example 217 is the apparatus of any of Examples 189 to 212, themaintenance task to comprise testing of one or more components of asled.

Example 218 is the apparatus of any of Examples 189 to 212, themaintenance task to comprise cleaning of one or more components of asled.

Example 219 is the apparatus of any of Examples 189 to 212, themaintenance task to comprise power cycling one or more memory modules.

Example 220 is the apparatus of any of Examples 189 to 212, themaintenance task to comprise power cycling one or more non-volatilestorage devices.

Example 221 is the apparatus of any of Examples 189 to 212, themaintenance task to comprise storing a compute state of a compute sled,replacing the compute sled with a second compute sled, and transferringthe stored compute state to the second compute sled.

Example 222 is the apparatus of any of Examples 189 to 212, themaintenance task to comprise replacing one or more cache memory modulesof a processor.

Example 223 is an automated maintenance device, comprising means foridentifying a collaborative maintenance procedure to be performed in adata center, means for identifying a second automated maintenance devicewith which to collaborate during performance of the collaborativemaintenance procedure, and means for sending interdevice coordinationinformation to the second automated maintenance device to initiate thecollaborative maintenance procedure.

Example 224 is the automated maintenance device of Example 223,comprising means for identifying the collaborative maintenance procedurebased on telemetry data associated with one or more physical resourcesof the data center.

Example 225 is the automated maintenance device of Example 224, thetelemetry data to include one or more telemetry metrics associated witha physical compute resource.

Example 226 is the automated maintenance device of any of Examples 224to 225, the telemetry data to include one or more telemetry metricsassociated with a physical accelerator resource.

Example 227 is the automated maintenance device of any of Examples 224to 226, the telemetry data to include one or more telemetry metricsassociated with a physical memory resource.

Example 228 is the automated maintenance device of any of Examples 224to 227, the telemetry data to include one or more telemetry metricsassociated with a physical storage resource.

Example 229 is the automated maintenance device of any of Examples 223to 228, comprising means for identifying the collaborative maintenanceprocedure based on environmental data comprising measurements of one ormore aspects of ambient conditions within the data center.

Example 230 is the automated maintenance device of Example 229,comprising one or more sensors to generate the measurements comprised inthe environmental data.

Example 231 is the automated maintenance device of any of Examples 229to 230, the environmental data to comprise one or more temperaturemeasurements.

Example 232 is the automated maintenance device of any of Examples 229to 231, the environmental data to comprise one or more humiditymeasurements.

Example 233 is the automated maintenance device of any of Examples 229to 232, the environmental data to comprise one or more air qualitymeasurements.

Example 234 is the automated maintenance device of any of Examples 229to 233, the environmental data to comprise one or more pressuremeasurements.

Example 235 is the automated maintenance device of Example 223,comprising means for identifying the collaborative maintenance procedurebased on an automation command received from an automation coordinatorfor the data center.

Example 236 is the automated maintenance device of Example 235,comprising means for identifying the collaborative maintenance procedurebased on a maintenance task code comprised in the received automationcommand.

Example 237 is the automated maintenance device of any of Examples 223to 236, comprising means for selecting the second automated maintenancedevice from among a plurality of automated maintenance devices in acandidate device pool for the data center.

Example 238 is the automated maintenance device of any of Examples 223to 237, comprising means for identifying the second automatedmaintenance device based on a parameter comprised in a command receivedfrom an automation coordinator for the data center.

Example 239 is the automated maintenance device of any of Examples 223to 238, the collaborative maintenance procedure to comprise replacing asled.

Example 240 is the automated maintenance device of Example 239, the sledto comprise a compute sled.

Example 241 is the automated maintenance device of Example 240, thecollaborative maintenance procedure to comprise removing the computesled from a sled space, removing a memory card from a connector slot ofthe compute sled, inserting the memory card into a connector slot of areplacement compute sled, and inserting the replacement compute sledinto the sled space.

Example 242 is the automated maintenance device of Example 241, thememory card to store a compute state of the compute sled.

Example 243 is the automated maintenance device of Example 242, thecollaborative maintenance procedure to comprise initiating a restorationof the stored compute state on the replacement compute sled.

Example 244 is the automated maintenance device of Example 239, the sledto comprise an accelerator sled, a memory sled, or a storage sled.

Example 245 is the automated maintenance device of any of Examples 223to 238, the collaborative maintenance procedure to comprise replacing acomponent on a sled.

Example 246 is the automated maintenance device of Example 245, thecomponent to comprise a processor, a field-programmable gate array(FPGA), a memory module, or a solid-state drive (SSD).

Example 247 is the automated maintenance device of any of Examples 223to 238, the collaborative maintenance procedure to comprise replacingone or more cache memory modules of a processor on a sled.

Example 248 is the automated maintenance device of Example 247, thecollaborative maintenance procedure to comprise removing a heat sinkfrom atop the processor, removing the processor from a socket tofacilitate access to one or more cache memory modules underlying theprocessor, removing the one or cache memory modules, inserting one ormore replacement cache memory modules, reinserting the processor intothe socket, and reinstalling the heat sink.

Example 249 is the automated maintenance device of any of Examples 223to 238, the collaborative maintenance procedure to comprise servicing acomponent on a sled.

Example 250 is the automated maintenance device of Example 249, thecollaborative maintenance procedure to comprise removing the sled from asled space of a rack.

Example 251 is the automated maintenance device of any of Examples 249to 250, the collaborative maintenance procedure to comprise removing thecomponent from the sled.

Example 252 is the automated maintenance device of any of Examples 249to 251, the collaborative maintenance procedure to comprise testing thecomponent.

Example 253 is the automated maintenance device of any of Examples 249to 252, the collaborative maintenance procedure to comprise cleaning thecomponent.

Example 254 is the automated maintenance device of any of Examples 249to 253, the collaborative maintenance procedure to comprisepower-cycling the component.

Example 255 is the automated maintenance device of any of Examples 249to 254, the collaborative maintenance procedure to comprise capturingone or more images of the component.

Example 256 is the automated maintenance device of any of Examples 249to 255, the component to comprise a processor, a field-programmable gatearray (FPGA), a memory module, or a solid-state drive (SSD).

Example 257 is the automated maintenance device of any of Examples 223to 256, the interdevice coordination information to comprise a rackidentifier (ID) associated with a rack within the data center.

Example 258 is the automated maintenance device of any of Examples 223to 257, the interdevice coordination information to comprise a sledspace identifier (ID) associated with a sled space within the datacenter.

Example 259 is the automated maintenance device of any of Examples 223to 258, the interdevice coordination information to comprise a slotidentifier (ID) associated with a connector socket on a sled within thedata center.

Example 260 is the automated maintenance device of any of Examples 223to 259, the interdevice coordination information to comprise a sledidentifier (ID) associated with a sled within the data center.

Example 261 is the automated maintenance device of any of Examples 223to 260, the interdevice coordination information to comprise a componentidentifier (ID) associated with a component on a sled within the datacenter.

Numerous specific details have been set forth herein to provide athorough understanding of the embodiments. It will be understood bythose skilled in the art, however, that the embodiments may be practicedwithout these specific details. In other instances, well-knownoperations, components, and circuits have not been described in detailso as not to obscure the embodiments. It can be appreciated that thespecific structural and functional details disclosed herein may berepresentative and do not necessarily limit the scope of theembodiments.

Some embodiments may be described using the expression “coupled” and“connected” along with their derivatives. These terms are not intendedas synonyms for each other. For example, some embodiments may bedescribed using the terms “connected” and/or “coupled” to indicate thattwo or more elements are in direct physical or electrical contact witheach other. The term “coupled,” however, may also mean that two or moreelements are not in direct contact with each other, but yet stillco-operate or interact with each other.

Unless specifically stated otherwise, it may be appreciated that termssuch as “processing,” “computing,” “calculating,” “determining,” or thelike, refer to the action and/or processes of a computer or computingsystem, or similar electronic computing device, that manipulates and/ortransforms data represented as physical quantities (e.g., electronic)within the computing system's registers and/or memories into other datasimilarly represented as physical quantities within the computingsystem's memories, registers or other such information storage,transmission or display devices. The embodiments are not limited in thiscontext.

It should be noted that the methods described herein do not have to beexecuted in the order described, or in any particular order. Moreover,various activities described with respect to the methods identifiedherein can be executed in serial or parallel fashion.

Although specific embodiments have been illustrated and describedherein, it should be appreciated that any arrangement calculated toachieve the same purpose may be substituted for the specific embodimentsshown. This disclosure is intended to cover any and all adaptations orvariations of various embodiments. It is to be understood that the abovedescription has been made in an illustrative fashion, and not arestrictive one. Combinations of the above embodiments, and otherembodiments not specifically described herein will be apparent to thoseof skill in the art upon reviewing the above description. Thus, thescope of various embodiments includes any other applications in whichthe above compositions, structures, and methods are used.

It is emphasized that the Abstract of the Disclosure is provided tocomply with 37 C.F.R. §1.72(b), requiring an abstract that will allowthe reader to quickly ascertain the nature of the technical disclosure.It is submitted with the understanding that it will not be used tointerpret or limit the scope or meaning of the claims. In addition, inthe foregoing Detailed Description, it can be seen that various featuresare grouped together in a single embodiment for the purpose ofstreamlining the disclosure. This method of disclosure is not to beinterpreted as reflecting an intention that the claimed embodimentsrequire more features than are expressly recited in each claim. Rather,as the following claims reflect, inventive subject matter lies in lessthan all features of a single disclosed embodiment. Thus the followingclaims are hereby incorporated into the Detailed Description, with eachclaim standing on its own as a separate preferred embodiment. In theappended claims, the terms “including” and “in which” are used as theplain-English equivalents of the respective terms “comprising” and“wherein,” respectively. Moreover, the terms “first,” “second,” and“third,” etc. are used merely as labels, and are not intended to imposenumerical requirements on their objects.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

What is claimed is:
 1. An automated maintenance device, comprising:processing circuitry; and non-transitory computer-readable storage mediacomprising instructions for execution by the processing circuitry tocause the automated maintenance device to: receive an automation commandfrom an automation coordinator for a data center; identify an automatedmaintenance procedure based on the received automation command; andperform the identified automated maintenance procedure in the datacenter.
 2. The automated maintenance device of claim 1, the automatedmaintenance procedure to comprise replacing a compute sled in the datacenter.
 3. The automated maintenance device of claim 2, the automatedmaintenance procedure to comprise: removing the compute sled from a sledspace within a rack; removing a memory card from a connector slot of thecompute sled, the memory card to store a compute state of the computesled; inserting the memory card into a connector slot of a replacementcompute sled; inserting the replacement compute sled into the sledspace; and initiating a restoration of the stored compute state on thereplacement compute sled.
 4. The automated maintenance device of claim1, the automated maintenance procedure to comprise replacing one or morecache memory modules of a processor on a sled.
 5. The automatedmaintenance device of claim 4, the automated maintenance procedure tocomprise: removing the processor from a socket to facilitate access toone or more cache memory modules underlying the processor; removing theone or cache memory modules; inserting one or more replacement cachememory modules; and reinserting the processor into the socket.
 6. Theautomated maintenance device of claim 5, the automated maintenanceprocedure to comprise: removing a heat sink from atop the processorprior to removing the processor from the socket; and reinstalling theheat sink after reinserting the processor into the socket.
 7. Theautomated maintenance device of claim 1, comprising a radio frequency(RF) interface to receive a wireless signal comprising the automationcommand.
 8. An apparatus for coordination of automated data centermaintenance, comprising: processing circuitry; and non-transitorycomputer-readable storage media comprising instructions for execution bythe processing circuitry to: identify a maintenance task to be performedin a data center; determine to initiate automated performance of themaintenance task; select an automated maintenance device to which toassign the maintenance task; and send an automation command to cause theautomated maintenance device to perform an automated maintenanceprocedure associated with the maintenance task.
 9. The apparatus ofclaim 8, the non-transitory computer-readable storage media comprisinginstructions for execution by the processing circuitry to identify themaintenance task based on telemetry data associated with one or morephysical resources of the data center.
 10. The apparatus of claim 8, thenon-transitory computer-readable storage media comprising instructionsfor execution by the processing circuitry to identify the maintenancetask based on environmental data received from one or more automatedmaintenance devices of the data center.
 11. The apparatus of claim 8,the non-transitory computer-readable storage media comprisinginstructions for execution by the processing circuitry to add themaintenance task to a pending task queue following identification of themaintenance task.
 12. The apparatus of claim 11, the non-transitorycomputer-readable storage media comprising instructions for execution bythe processing circuitry to determine to initiate automated performanceof the maintenance task based on a determination that the maintenancetask constitutes a highest priority task among one or more maintenancetasks comprised in the pending task queue.
 13. The apparatus of claim12, the non-transitory computer-readable storage media comprisinginstructions for execution by the processing circuitry to select theautomated maintenance device from among one or more automatedmaintenance devices in a candidate device pool.
 14. A method forautomated data center maintenance, comprising: receiving, at anautomated maintenance device, an automation command from an automationcoordinator for a data center; identifying, by processing circuitry ofthe automated maintenance device, an automated maintenance procedurebased on the received automation command; and performing the identifiedautomated maintenance procedure in the data center.
 15. The method ofclaim 14, the automated maintenance procedure to comprise replacing acompute sled in the data center.
 16. The method of claim 15, theautomated maintenance procedure to comprise: removing the compute sledfrom a sled space within a rack; removing a memory card from a connectorslot of the compute sled, the memory card to store a compute state ofthe compute sled; inserting the memory card into a connector slot of areplacement compute sled; inserting the replacement compute sled intothe sled space; and initiating a restoration of the stored compute stateon the replacement compute sled.
 17. The method of claim 14, theautomated maintenance procedure to comprise replacing one or more cachememory modules of a processor on a sled.
 18. The method of claim 17, theautomated maintenance procedure to comprise: removing the processor froma socket to facilitate access to one or more cache memory modulesunderlying the processor; removing the one or cache memory modules;inserting one or more replacement cache memory modules; and reinsertingthe processor into the socket.
 19. The method of claim 18, the automatedmaintenance procedure to comprise: removing a heat sink from atop theprocessor prior to removing the processor from the socket; andreinstalling the heat sink after reinserting the processor into thesocket.
 20. At least one non-transitory computer-readable storage mediumcomprising a set of instructions that, when executed by an automationcoordinator for a data center, cause the automation coordinator to:identify a maintenance task to be performed in a data center; determineto initiate automated performance of the maintenance task; select anautomated maintenance device to which to assign the maintenance task;and send an automation command to cause the automated maintenance deviceto perform an automated maintenance procedure associated with themaintenance task.
 21. The at least one non-transitory computer-readablestorage medium of claim 20, comprising instructions that, when executedby the automation coordinator, cause the automation coordinator toidentify the maintenance task based on telemetry data associated withone or more physical resources of the data center.
 22. The at least onenon-transitory computer-readable storage medium of claim 20, comprisinginstructions that, when executed by the automation coordinator, causethe automation coordinator to identify the maintenance task based onenvironmental data received from one or more automated maintenancedevices of the data center.
 23. The at least one non-transitorycomputer-readable storage medium of claim 20, comprising instructionsthat, when executed by the automation coordinator, cause the automationcoordinator to add the maintenance task to a pending task queuefollowing identification of the maintenance task.
 24. The at least onenon-transitory computer-readable storage medium of claim 23, comprisinginstructions that, when executed by the automation coordinator, causethe automation coordinator to determine to initiate automatedperformance of the maintenance task based on a determination that themaintenance task constitutes a highest priority task among one or moremaintenance tasks comprised in the pending task queue.
 25. The at leastone non-transitory computer-readable storage medium of claim 24,comprising instructions that, when executed by the automationcoordinator, cause the automation coordinator to select the automatedmaintenance device from among one or more automated maintenance devicesin a candidate device pool.